AR headset with an improved display

ABSTRACT

Augmented reality headsets. A plurality of tilted pin-mirrors imbedded between an inner surface and an outer surface of a combiner, where the plurality of tilted pin-mirrors are configured to reflect the guided image light towards the eye box, and wherein the plurality of pin-mirrors include one or more gaps between them wherein the one or more gaps allow the passage of an ambient light through the combiner towards the eye box.

BACKGROUND

As computer technology is migrating in sophistication, complexity, powerand realism, one could say that the ultimate goal is to create acomputerized human being. As this process is unfolding before our eyes,the humans are not sitting idly by just watching, but rather, they arealso taking steps toward entering a computerized world. We have seenthis in the distance past with the creation of the Six Million DollarMan as well as the migration of Sci-Fi movies like the Matrix and ReadyPlayer One. Maybe someday we will live in a world where the computer andmankind are fully joined, but in the meantime, the human venture intothe computer world is being played out in the virtual reality andaugmented reality technologies.

Virtual reality (VR) is an interactive computer-generated experiencetaking place within a simulated environment. This simulate environmentoften includes audio and visual elements, as well as other elements suchas sensory feedback (vibrations, motion, smells, temperature and touch(haptics)). The VR immersive environment can be similar to the realworld or it can be fantastical, creating an experience that is notpossible in ordinary physical reality.

Augmented reality (AR) systems may also be considered a form of VR. Themain different between AR and VR is that AR layers virtual informationover a live camera feed or actual visualization of one's environmentwith the eye giving the user the ability to view three-dimensionalimages integrated into his or her real world.

At present, there are two primary architectures for implementing ARglasses. In a first version, image light is incident to the inside faceof a curved combiner and then re-directed towards the eye box. In orderto provide nominally collimated image light to the viewer for all fieldpoints, the combiner can have an extreme shape, particularly if a largefield of view (FOV) is sought. In such systems, it can also be difficultto fit the optics on the temples, next to the head, while having theimage light rays steer clear or miss hitting the side of the face. In asecond version, such as the MICROSOFT HOLOLENS, the image light isdirected into an edge of a flat waveguide or light guide, through whichit propagates, until it is extracted or redirected by output couplingoptics towards a viewer's eyes. Use of such waveguides canadvantageously reduce the volume needed for the optics, but thediffraction gratings used for light coupling can create both chromaticand stray light image artifacts. Also, at present, the migration to ARis plagued with limitations, including the cost of the equipment, thesize, bulkiness or weight of the equipment, and the limitedfunctionality of the equipment.

Another problem that is particularly evident in AR is that with thereal-world images combined with the virtual images, a user may havetrouble focusing. From an optical perspective, everyday objects are amyriad of points of light-emitting rays that, after penetrating thepupil of the eye, form an image on the retina. According to the laws ofgeometrical optics, when the optical system of the eye is well focused,each point of light in the object forms a point of light in the retinalimage. In reality, the image is not a simple point, because the physicalfactors of diffraction and interference distribute the light across theretina.¹ ¹ Larry N Thibos, Cameron A Thibos US Ophthalmic Review 2011;4(2):104-6 DOI: http/doi.org/10.17925/USOR.2011.04.02.104

When the optical system of an eye 100 is mis-focused on an object 115,the image of any single point of light is uniformly spread out across asmall area of retinal surface. As illustrated in FIG. 1, the shape ofthe pupil 110 of the eye 100 determines the shape of the blurred retinalimage. Given that the shape of the pupil 110 in the normal human eye 100is circular, the image is a circular region called a ‘blur circle’ 120or ‘blur disk’. By comparison, the eye of a cat has a verticallyelongated pupil, so the retinal image would be a ‘blur ellipse’. Thehuman pupil also takes on an elliptical appearance when viewed from theside, so the blurred image in peripheral retina is also a ‘blurellipse’. Some animals have a pupil that forms two small pinholes, whichwould produce a pair of small blur disks for every object point, anatural example of monocular diplopia.² ² IBID

Pinhole glasses, also called stenopeic glasses, are eyeglasses withlenses that consist of many tiny holes filling an opaque sheet ofplastic. These “pinholes” block indirect rays from entering the eye,thus preventing them from distorting your vision. While this does notactually improve the focusing ability of the eye, it does reduce thesize of the blur circle on the back of the retina, so reasonably clearvision may be achieved.3 However, while viewing through a single pinholecan improve resolution, within the trade-off of aberration blur versusdiffraction blur, the resulting image will be dim. By comparison,stenopeic glasses with multiple pinholes increase the vision angle andthe amount of light that reaches the retina. If two pinholes areseparated by less than the diameter of the pupil aperture, two pencilsof ray coming from one light point pass through the pupil and form twonearby retinal images. Optimization is necessary, as using too large ofa separation will result in dead spots in the field, while too small ofa separation will produce multiple images. ³ Using Pinhole Glasses forVision Improvement,https://www.verywellhealth.com/do-pinhole-glasses-work-3421901 By TroyBedinghaus, OD Updated Oct. 8, 2017

Thus, there are yet opportunities for improved wide FOV AR glasses orheadsets that have better optical designs and performance, includingapproaches that provide enhanced resolution or smaller blur circles.

SUMMARY

The present disclosure is related to augmented reality headsets and moreparticularly light guided augmented reality (AR) display that include animage source and imaging optics. The imaging options provide an imagelight. The AR display also includes a combiner into which the imagelight is end or edge coupled, and from which the image light is guidedand output towards an eye box. A plurality of tilted pin-mirrorsimbedded between an inner surface and an outer surface of the combiner,where the plurality of tilted pin-mirrors are configured to reflect theguided image light towards the eye box, and wherein the plurality ofpin-mirrors include one or more gaps between them wherein the one ormore gaps allow the passage of an ambient light through the combinertowards the eye box. With respect to the image light, the tiltedpin-mirrors appear to form a high fill factor array, whilesimultaneously appearing as a low fill factor array for ambient lightincident to an outer side surface of the combiner.

In exemplary embodiment is an augmented reality headset (ARHS) displaythat incorporates a scanning projection image source, The ARHS includesa tri-linear display with multiple rows of pixels for rendering apixelated image source. The ARHS also includes imaging optics whichprovide image light to a combiner. A scanning system within the ARHSoperates to deflect the image light along one axis. A control systemoperates to temporally vary the image shown on each row of pixels of themultiple rows so as to increase the perceived brightness of the image.

In some embodiments, the scanning system is a 1D scanning mirror withreciprocating motion. In other embodiments, the scanning system is apolygonal mirror rotating with uniform velocity. In yet otherembodiments, the scanning system is an electro-optic beam deflectiondevice.

In the various embodiments, the tri-linear display may include aplurality of monochromatic blocks of pixels. Further, in someembodiments the intensity of the pixels in each of the multiple rows ofpixels in the tri-linear display can be modulated and/or the number ofrows of pixels in the tri-linear display which are lit can be adjustedto control a grey level.

In other embodiments, the tri-linear display may include one or moreblocks comprising a plurality of pixels. In such embodiments, to attaina lower luminous output for a particular block the number of pixels in aparticular block are increased.

The control system within the ARHS may implement a pixel illuminationscheme. As such, when a reduction in the illumination in a column ofpixels as the result of a defective pixel in the column occurs, thescheme compensates for this by illuminating at least one additionalpixel in the column.

In yet other embodiments of the ARHS, the trilinear display may includea plurality of monochromatic blocks. In such embodiments, each of theplurality of monochromatic blocks includes a plurality of rows. Each ofthe plurality of rows are alternately offset from adjacent rows by afractional pixel pitch so as to enhance system resolution. In someembodiments, the pixels are masked with an opaque mask so as to reduce afill factor. In other embodiments, the ARHS comprises a softwareanti-aliasing filter that is configured to deconvolve against a displaybox filter to create a modified anti-aliasing filter.

In even further embodiments, the tri-linear display comprises more thanthree types of color emitters. The more than three types of coloremitters expand the color gamut of the tri-linear display.

In the various embodiments of the ARHS, the combiner may include animbedded plurality of pin-mirrors that reflect image light towards aneye box. Gaps may exist between the pin-mirrors in the plurality ofpin-mirrors to allow ambient light to pass through the combiner to theeye box. In some embodiments, the pin-mirror size, as seen by a viewer,is nominally constant across the combiner. Further, in some embodiments,the size, shape, spacing, or tilt of the pin-mirrors can be adjusted tochange the optical efficiency of the combiner directing the image lightto the eye box. Further, in some embodiments, the size, shape, and tiltof the pin-mirrors can be adjusted to change an image blur. Evenfurther, in some embodiments, the size, shape, spacing, or tilt of thepin-mirrors can be adjusted to change the optical efficiency of allowingambient light to pass through the combiner to the eye box.

In some embodiments of the ARHS, a coating of the pin-mirror isconfigured to improve an optical efficiency of a transiting image lightor the ambient light. In other embodiments of the ARHS, the plurality ofpin-mirrors is arranged into multiple sub-arrays.

Disclosed embodiments also include method for providing an image from anARHS. The method includes the action of providing an ARHS with a pair ofdisplays, each of the pair of displays comprising a pixelated colordisplay that provides an image light into a corresponding projectionoptics. Further, the method continues by directing the image lightthrough the corresponding projection optics and edge coupling the imagelight into a lightguide combiner, wherein the lightguide combiner isconfigured to redirect transiting image light towards a correspondingeye box. In such embodiments, the light guide combiner includes animbedded plurality of pin-mirrors that reflect the image light towardsthe corresponding eye box, and wherein one or more gaps between thepin-mirrors allow ambient light to pass through the combiner to thecorresponding eye box. Further, with respect to the image light, thepin-mirrors appear to form a high-fill factor array, whilesimultaneously appearing as a low-fill factor array for ambient lightincident to an outer side surface of the combiner.

These and other embodiments, features, aspects and benefits aredescribed more fully in the detailed description with the accompanyingdrawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 depicts optical and imaging effects of pinhole glasses.

FIG. 2 depicts the concept of virtual images.

FIG. 3A depicts an exemplary projection type AR headset.

FIG. 3B depicts an exemplary light guide type AR headset.

FIG. 4A depicts a concept for pin mirror arrays for use in AR headsets.

FIG. 4B depicts a concept for pin mirror arrays for use in AR headsetsin a stacked configuration.

FIG. 5A depicts a second concept for pin mirror arrays for use in ARheadsets.

FIG. 5B presents a cross-sectional view of the pin mirror array of FIG.5A taken line 5B-5B.

FIGS. 6A and 6B depicts potential orientations of the pin-mirrors in apin-mirror array.

FIG. 7A depicts side view of portions of an improved light-guide type ARheadset, including image light propagation through a combiner or eyepiece with a pin-mirror array.

FIG. 7B depicts a side view of the embodiment of FIG. 7A with moredetail showing the slice edges.

FIG. 7C depicts top side view of portions of an improved light-guidetype AR headset of FIG. 7A.

FIG. 7D depicts top front view of portions of an improved light-guidetype AR headset of FIG. 7A.

FIG. 8 depicts a cross-sectional view of the pin-mirrors within acombiner, to illustrate a spatial variation of the pin-mirror tilt.

FIGS. 9A-E depict a second improved approach for a pin-mirror basedlight guide AR headset having a dual light guide and a curved reflector.

FIG. 10A-C depicts different views of a viewer's eye receiving virtualimage light from part of an AR headset having pin-mirrors.

FIG. 11 depicts an optimization method for designing combiners oreyepieces for AR headsets having a plurality of pin-mirrors.

FIG. 12 depicts a third improved AR headset, of the projection type,having a plurality of pin-mirrors.

FIG. 13 depicts aspects of the construction of a combiner having aplurality of pin-mirrors.

FIG. 14 depicts a fourth improved AR headset, of the light-guide type,having a scanning image light source and a plurality of pin-mirrors.

FIG. 15 depicts a portion of a pixelated tri-linear image source for usein providing a scanning image light source.

FIG. 16 depicts a portion of an ARHS with a modified trilinear displaywith TDI operation and a scanner.

FIG. 17 depicts a distribution of light in an eyebox for a single pixelfor a ARHS with a pin-mirror based combiner.

FIG. 18 depicts portion of a pin-mirror based light combiner and aspatial variation in intensity across a virtual image.

FIG. 19 depicts the path taken by light from a single pixel in theimage, and its interaction with the pin-mirror based light guidecombiner structure.

FIG. 20 depicts the interaction of image light from a single pixel withthe pin-mirror based light guide combiner layers.

FIG. 21 depicts the footprint of the light reflected from a portion ofthe combiner layers into an eyebox.

FIG. 22 depicts a portion of a pin-mirror based light guide combinerwith an arbitrary non-elliptical optimized reflective pattern of thepin-mirrors.

FIG. 23 depicts an improved light guide combiner with a segmented layerstructure.

FIG. 24 depicts a general form of a discrete geometric pin-mirror basedlightguide type combiner.

FIG. 25 depicts a modified trilinear display with the light emittingpixels having a staggered row structure.

FIG. 26 depicts an improved modified light emitting display with anopaque mask to reduce apparent pixel fill-factor.

FIG. 27 depicts a portion of an improved light emitting display with asubaperture microlens array to reduce apparent pixel fill-factor.

FIG. 28 depicts a portion of an improved light guide combiner based ARHShaving projection optics with a light emitting display with a tiltedmicrolens array.

FIG. 29 depicts a portion of improved projection optics.

FIG. 30 depicts a portion of improved projection optics having a foldedcatadioptric system that operates on polarized light.

FIG. 31 depicts a portion of improved projection optics for a 1Dscanning ARHS further including scanner position sensing.

FIG. 32 depicts a portion of improved optics for an ARHS having a lightguide combiner.

FIG. 33 depicts how a scanning trilinear projection system can replace aprojection system which utilizes an area display.

FIG. 34 depicts an ARHS display utilizing a scanning trilinearprojection system in conjunction with a free-space combiner.

FIG. 35A and FIG. 35B (collectively FIG. 35) depict an ARHS displayutilizing a scanning trilinear projection system in conjunction with afree-form prism combiner.

FIG. 36 depicts an ARHS display utilizing a scanning trilinearprojection system in conjunction with a holographic combiner.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

The present invention, as well as features and aspects thereof, isdirected towards providing an optical solution that utilizes pin-holetechnology to reduce the blur circle for AR solutions, such as ARHeadsets (ARHS).

A virtual image is an image that from the user's perspective, is notprojected on a screen but rather appears to be present in space. Thus,in an AR system, virtual images are generated to give the appearance ofexisting in the user's real-world space.

A good tutorial of this field of art can be found in the United Statespublished patent application US20100290127A1, which is summarized in thenext few paragraphs.

A virtual image is different from a real image and the images are formeddifferently as well. A real image is an actual image that can beobserved directly by the unaided human eye. A real image is present inthe real world and the image is perceived by the human eye when lightbouncing off of the image enters into the eye through the pupil andlands on the retina wall within the eye. Thus, a real image is aperception of a physically existing object at a given location. Anexample of a real image is a photograph. Real images can be createdelectronically with devices such as cathode ray tubes (CRT), liquidcrystal displays (LCD) screens, liquid crystal on silicon (LCOS)devices, digital micro-mirror display devices (DLP or DMDs), lasers,super luminescent diodes (SLEDs), and organic light emitting diode(OLED) displays. The OLED is an example of an electronic display thatprovides a real image. The size of the display surface limits the sizeof the real image that can be provided to the observer.

Virtual image displays provide an image that is not observable on aphysically existing viewing surface or in a tangible world. The virtualimage is formed at a location in space where no display surface exists.An example of creating a virtual image is when someone looks at smallitems through a magnifying glass. The magnifying glass makes the imageappear larger and the image also appears to be located substantiallybehind the surface where the item actually exists. Thus, while the itemis a real image, the magnification of the item is a virtual image. Bydefinition, a virtual image can exist at a location where no displaysurface exists. The size of the virtual image therefore is not limitedby the size of a display surface. Virtual image electronic displays thushave the advantage of eliminating the need for a large display surfacein order to produce a large electronic image.

FIG. 2 illustrates how a virtual image can be created by viewing anobject 202 through a magnifying lens 204. The object 202 is placedwithin the focal length 210, or f, of a magnifying lens 204. The virtualimage 206 that is formed appears to the viewer at point 208 and isenlarged and has the same orientation as the source object 202. As aresult of this type of image formation, the size of the virtual image206, as perceived by the viewer 212, is limited by the magnification ofthe display system as opposed to the size of the electronic display.This enables virtual image displays to be designed that provide the sameamount of information per screen as real image displays, yet occupy asmaller space.

Thus, it can be appreciated that an optical system is used to create avirtual image. As such, the eye and the viewing surface properties of areal image are the factors that determine the viewing parameters,whereas in a virtual image display, the optical system determines mostof the viewing parameters.

In the creation of an AR environment, especially one that is createdthrough the use of a viewing headset, light waves enter the pupil of theeye from the real environment as well as from the virtual imagegenerating optic system. In an AR headset (ARHS), a real image thatserves as the source object is first formed by an imaging component thatis electronically energizable to form an image from image data. Inembodiments of the present invention, an OLED or other emissive displaydevice is utilized to create a real image and then, a virtual image isthen created through an optical system. Obviously, within an ARHS, theimaging source needs to be small and inexpensive in order to reduce thesize and overall cost of the ARHS. But it should be understood thatwhile OLEDs can be utilized, other image sources may also be utilized,such as LCDs, etc. The optic system then forms a virtual image of thereal image generated by the source, or OLED in the describedembodiments. The virtual image is then seen by the viewer along with theactual real world in which they are located.

FIG. 3A depicts a portion of an exemplary projection type AugmentedReality headset 300, in which incident light reflects off the inner orinside surface of a combiner, towards a viewer's eye. This ARHS displayincludes a frame 302 with a right arm or temple 304 and a strap 306 thatcan be used to secure the ARHS 300 to a user's head. A comparable system(not shown) is typically provided for the left eye. Depending on theweight of the ARHS, the strap 306 that extends over the top of the headand/or behind the neck may or may not be needed to provide additionalsupport. The headset rests on a viewer's nose with enough offset, or eyerelief, of the combiners from the viewer's eyes so as to be comfortable.

In particular, as shown in FIG. 3A, a projection type ARHS can includevirtual image generating optics 310 on the right temple 304. The imagegenerating optics 310 can include a LED display 312, imaging optics 314,and a variable opacity combiner 316, that together provide and directvisible AR image light to a viewer's eye. In particular, the image lightprovided by the image generating optics that is directed towards thevisor or combiner 316, hits the inner surface 318 of the combiner,between a temple 304 and the nose bridge 305, and reflects back towardsthe user's eye, or a target area or eye box 320 nominally overlappingwith the expected eye position. An eye box is defined as the volume ofspace within which an effectively viewable image is formed by the ARHSdisplay, and it represents a combination of an exit pupil size and aneye relief distance. Nominally, the exit pupil size is assumed to bethat of human viewer's experiencing photopic light levels (e.g., ≥1.0cd/m2). Whereas, an eye relief is the distance from the last surface ofan eyepiece within which the viewer's eye can obtain the full viewingangle. The visor or combiner 316 is curved and shaped to fit well, orconform to, the shape or contours of a viewer's face.

A second set of image generating optics (not shown in FIG. 3A) likewisecan provide image light to a viewer's left eye. The combiner 316, oreyepiece, is referred to as having a variable opacity in part as theamount of ARHS image light that is seen by a viewer depends on thereflectivity of the dichroic coating provided on the inner surface 318of the combiners 316, while the transmissivity of the combiners alsohelps determine the amount of ambient or environmental light thatreaches a viewer's eyes and the amount of ARHS image light that is lostthrough the glasses into the ambient environment. Nominally, thedichroic coatings of the combiner are nominally 50% reflective for thevisible spectrum, but the reflectivity or transmissivity of thecombiners 316 can also vary spatially and angularly over the surface ofthe combiners.

A typical ARHS 300, such as the Microsoft “Hololens”, can provide imagecontent to a limited FOV per eye of <50o. Ideally, for some viewingapplications, a projection type ARHS 300 would support a WFOV per eye inexcess of 90o, to as much as 115o or larger, and thus provide content tothe user's peripheral vision. In projection type ARHS glasses, the imagegenerating optics 310 would produce a fan of rays that spans anasymmetrical FOV ≥30o in total width, that then intersects with a highlycurved combiner (e.g., radius ≥60 mm) having a compound or complex(e.g., aspheric) curvature, to produce nominally “collimated” ray fansdirected over a large FOV to an eye box 320 of ≥10 mm in width. Thistype of ARHS can be difficult to design and make because of the space,weight, and image quality constraints, as well as the difficult designand fabrication specifications for the optics.

FIG. 3B depicts another exemplary type of AR headset 350. In this case,image light from virtual image generating optics 360 is directed into aninput coupler 362 and into a wave guide or light guide 364. The imagelight can then exit towards a viewer's eye via an output coupler 366.The input coupler, which is positioned at or near an end of the lightguide, can be a prism, diffraction grating, or edge-lighting mechanism.The light guide for an optical waveguide display is a sheet oftransparent material with two surfaces, which are locally parallel andoptically polished.

In this ARHS display, the coupled wave or image light is confined insidethe waveguide or light guide through total internal reflection (TIR) onthe waveguide surfaces and propagates along following a zigzag TIR lightpath 368. As shown in FIG. 3B, the projection or image generating optics360, and pre-input coupler 362 edge couple “collimated” image light intothe plane of the light guide, and convergent focused image light in theplane vertical to the light guide. In the latter case, the image lightcan be convergent at ˜F/4, at a distance just inside (from the inputcoupler) the waveguide of about 23 mm, with a tilt beyond the criticaldegree, such as ˜70 degrees from the surface normals. The image lightcan then TIR its way through the light guide(s) to the diffractiongrating output coupler(s). A hologram or diffraction grating is placedparallel to and immediately in contact with the waveguide. When thegrating is illuminated with the guided wave, the image light is directedto a viewer's eye, enabling the viewer to see a virtual image.

In one version, the complete FIG. 3B system has three light guidesarranged in parallel; one per color (RGB). Similar systems, with one orthree waveguides, can be used to provide image light to a viewer's othereye. As compared to the ARHS 300 of FIG. 3A in which image light isdirected onto the highly curved inner surface 318 of the combiner 316,the use of waveguides can enable a smaller ARHS 350 with simpler imagingoptics. The Microsoft “Hololens” glasses are one commercially availableexample of this type of glasses.

However, projecting image light in and out of this light guided systemintroduces its own set of problems, particularly with regards to the“diffraction gratings”. As the waveguides are heavily wavelengthdependent, this type of glasses is typically made with three separatewaveguides with three separate sets of gratings to handle each of theRGB colors. This separation, along with their display system, can leadto a user seeing the virtual images separated into their colorcomponents. Another side effect for the lightguide type of ARHS, is thatwhen a viewer's head is moving, a white spot (combined by R+G+B) can beseen as separated into three separated RGB spots. A further side effectis that when a user is directly viewing a bright light source from theambient real environment, the diffraction grating can cause some ghostimages. Also, the gratings can also redirect and disperse stray ambientlight and create additional artifacts, not related to either thedisplayed virtual image or the real scene.

A variety of improved AR headsets using novel pin-mirror basedcombiners, which are also referred to as eyepieces or visors, aredisclosed. According to an embodiment of the present invention, as shownin FIG. 4A, a novel type of improved light guide combiner 405 can bemade of adjacent or interlocking slices 410 of transparent glass orpolymer, with at least one row or one dimensional (1D) array or lineararray of reflective surfaces or pin-mirrors 430 to form part of apin-mirror array 435 that are fabricated along tilted edge faces orfacets 425 of a slice 410. The nominally preferred facet tilt angle Ø412is 45° relative the flat sides 411 of the slices 410 as a non-limitingexample but it is appreciated that other angles may also be utilized. Aseries of pre-fabricated slices 410 can then be assembled along thex-axis to form a larger flat combiner 406 providing a two-dimensional(2D) pin-mirror array of imbedded tilted reflectors (pin-mirrors 430).The individual small mirror areas in FIG. 4B, which are positionedrelative to another by a center to center pitch 432′ and 432″(collectively referred to as 432), or are separated from each other byedge to edge gaps, can be optimized to be spaced with nominally the sameor different pitch or gaps in the horizontal and vertical directions.Although the pitch 432′ and 432″ varies by design, and can varyspatially within a design, the pitch 432′ and 432″ can be equal ordifferent, and is most commonly is in 4-6 mm range. The pin-mirrors 430are reflectors, which can be provided with a metal coating, such as ofaluminum or silver, or as a multilayer dielectric or dichroic thin filmcoating, and for example, provide ˜80-96% local reflectivity, dependingon the coating type. Although the pin-mirrors 430 are nominallyfabricated with broadband visible reflective coatings, notch typedichroic coatings, that reflect a set of narrow (e.g., ≤25 nm wide) RGBspectral bands can also be used. As these notch type coatings can allowambient light outside of the narrow reflection bands to pass through thepin-mirrors to the eye box, the optimization of the efficiency of theambient light transmission can be desensitized relative to theoptimization of the size and spatial density of the pin-mirrors 430.

Considering FIG. 4A and FIG. 4B in greater detail, the combiner 405 canbe made with interlocking slices 410 of glass or plastic, whose tiltededge facets 425 are coated partially or completely using materials toprovide a partially transparent or completely reflective surface. Themirrored facets can then be coated with an index-matching adhesive, andstacked sequentially to form a larger combiner 406, for which thenon-mirrored surfaces form uniform, optically-continuous segments. Forgreater mechanical stability, the slices 410 can also be assembled ontoa larger substrate 440, which can be provided on either side of thecombiner 406, towards or away from the viewer's eyes. Once a combiner isassembled with imbedded micro-mirrors or pin-mirrors 430, the outersurfaces of the combiner 406 can also be fabricated with anti-reflection(AR) coatings or other coatings to improve the optical quality of theassembly.

As another example, an improved combiner with pin-mirrors 430 can befabricated by cutting a row of grooves partially into a substratematerial to provide the tilted edge facets 425. The pin-mirrors 430 ormicro-mirrors can then be deposited or coated in either a linear or 2Darray within the grooved facets 425. To imbed the pin-mirrors 430, thegrooves can then be filled with an inserted compensating piece ofsubstrate material, or with adhesive, or with a 3D printed volume ofindex matched material. Alternately, a second substrate with matchingprotruding faceted ridges can be overlaid on the first substrate, so theridges fill the faceted grooves. The two substrates can be fusedtogether, or attached by adhesive, or a second equivalent substrate canbe cast or molded over the first one. As a further alternative, thereflective coatings for the pin-mirrors 430 or micro-mirrors can bedeposited or formed on protruding ridges provided on a substrate, and asecond grooved substrate, or a set of slices, can be used to fill in thespaces between the ridges, and to create an overall combiner 405 withsmooth external surfaces (inner surface 427 and outer surface 428).

FIG. 5A depicts a second concept for pin mirror arrays for use in ARheadsets. FIG. 5B presents a cross-sectional view of the pin mirrorarray of FIG. 5A taken line 5B-5B. As such, an alternate configurationfor the improved combiner is depicted in FIG. 5, in which multiple rowsof tilted reflectors or pin-mirrors 530 are fabricated on a tilted facet525 of a slice 510 to form part of a pin-mirror array 537. Multiple suchslices can then be assembled into part of a larger combiner 505providing a larger 3D pin-mirror array 537 of tilted pin-mirrorsub-arrays 535 of pin-mirrors 530 are provided on an imbedded surfacewithin a combiner 505. While FIG. 5 depicts this type of combiner oreyepiece before the slices and sub-arrays are assembled into thecombiner, FIG. 14 depicts an improved ARHS (1400) with an assembledcombiner of this type, including a pitch 1432 between sub-arrays. Avariety of related parameters, including the extent of the sub-arrays535, the pitch 532 between the pin-mirrors, the gaps 540 and gaps 545between the pin-mirrors, the parallelism or relative skew or tiltbetween the sub-arrays, the curvature of the sub-arrays, and thepin-mirror patterning within or between the individual sub-arrays(including in-plane offsets), can be optimized. For example, thepin-mirror pitch or a pin-mirror sub-array positioning or offset can bevaried spatially from one sub-array to the next, so as to optimize theapparent fill-factor for the transiting image light.

Also, as presented in example in FIGS. 6A and 6B, the orientation of therows of pin-mirrors 630A can seem to be in a row, or a tilted row 630Bdepending on how the assembled part is viewed. This is made clearer inFIG. 7(A-D), where an improved light guide type AR headset 700 is shownin cross-section, and both the side and end views of a combiner 705 aredepicted along with illustrations of propagating image rays. As shown inFIG. 7, as seen from the inner or outer surface, an assembled combiner705 looks like a nominally transparent member with a 2D array ofpin-mirrors 730. Whereas, as seen best in the side view, the assembledcombiner looks like a narrow structure with imbedded angled mirrors 730.When a combiner 705 is assembled using these slices, the overallcombiner structure can provide several “columns” of reflectors, as longas the rows or columns don't block each other (see FIG. 6B). This is incontrast to the left image (FIG. 6A), where the pin-mirrors 630 couldblock each other, in terms of the paths to an eye box 720. As seen onFIGS. 6A and 6B, the number of pin mirror layers along x-z plane isdependent on the thickness of the eyepiece. Two or three layers can bepreferentially selected if the nominal diameter of the pin mirror isoptimized to be 2 mm, as a non-limiting example.

FIG. 7A is a top-planar view of an ARHS visor 700 that with a pin-mirrorbased combiner 705 and virtual image generating optics 750, can beincorporated into an improved AR headset display. The visor 700 receiveslight rays 760 from the real-world environment as well as light rays 745from a virtual image source 750. The virtual image generating optics 750can include an LED or OLED array with a 2D array of light emittingpixels that can emit light in the F/1.5-F/2.5 range. These optics canfurther include spherical or cylindrical optical elements (e.g., lenses780 or mirrors) to modify the divergent image light, and nominallycollimate this light from a given pixel in at least one direction, andto then direct the light into a visor or lightguide (see FIGS. 9A-E forfurther illustration of lens elements 780). The visor 700 includes aseries of embedded mirrors or pin mirrors 730 that are embedded betweenan external inner surface 727 and an external outer surface 728 of thevisor 700. The beam from each pixel on the image source is nominallycollimated by the projection optics and then coupled into the lightguideeyepiece (705), after which is reflected by the embedded mirrors or pinmirrors 730 and coupled out towards an eye box 722 to be visualized by ahuman eye. As image light from a display pixel interacts with one ormore pin-mirrors 730, and light from other display pixels interacts withother pin-mirrors, an aggregate light beam can be directed to the eyebox 720. The aggregate light forms a convergent light cone directed toan eye box 720, but image light for any one image pixel is nominallycollimated into the eye box. The optics of a viewer's eye can then alteror focus this light so that an image can be perceived. The apparentfield of view depends on the angular width of the aggregate cone oflight directed into the eye box 720. Although FIG. 7A does not showdetails of light coupling into the light guide combiner, the preferredtechniques are edge coupling, a coupling prism (as in FIG. 4), or usingan edge facet (FIGS. 9A-E).

In this display device, image light rays 745 can be coupled into an edgeof an improved light guide combiner 705, or eyepiece, from one or morespecified input facets or input couplers, and after propagating througha light guide portion 720 of the combiner 705, be reflected by theimbedded pin-mirrors 730 and projected outward towards a viewer's eye.Depending on the directionality of the incoming image light,corresponding to a given image pixel, the light will propagate a shorteror longer distance along the length of the light guide, beforeinteracting with one or more pin-mirrors 730. As shown in FIG. 7A, thelight can be directed through the light guide in part by total internalreflection (TIR). FIG. 7A depicts the image light as bouncing once andreaching a target pin-mirror 730, but TIR provides that the light istrapped and can bounce multiple times within a flat light guide portionof the combiner, and likely will do so, around or past part of the arrayof pin-mirrors before reaching its target pin-mirror(s) 730.

With respect to FIG. 7A, the pin-mirrors 730 are nominally sized to havea real sub-pupil ˜0.3-2 mm full width, as compared to the nominalphotopically adapted adult maximum pupil width of ˜4 mm. A goal is thatthe individual pin-mirrors 730 are too small for viewers to really seeor focus on while the AR glasses are being worn. Also, while thepin-mirrors 730 can be rectangular in shape, as suggested in FIGS. 4A-B,the pin-mirrors 730 are preferentially fabricated to be circular orelliptical in shape, as depicted in FIG. 7C-D. Then, taking the nominal45-degree tilt of the imbedded facets 725 into account, the nominallyflat pin-mirrors can have apparent sizes that can be as small as ½ thereal size, and a goal is to have the pin-mirrors 730 appear circular toa viewer's eye. Thus, pin-mirrors 730 having a real elliptical shape, asshown in FIG. 7D, can have a circular appearance with respect to aviewer when tilted. The pin-mirror shape can also be more irregular.

As also shown in FIG. 7D, the pin-mirrors 430 are offset from one toanother by fairly large gaps within a pin-mirror array 735, that in thedirection of light propagation can be as large as 6-10 mm wide, or about3-5× the width of the pin-mirrors 730. By comparison, for theorientation of the combiner that is orthogonal to the direction of raypropagation, the real gaps can again be about 7-10 mm wide, but as seenfrom direction of the virtual imaging generating optics 750, thestaggered pin-mirror-arrays 735 appear to have little to no apparentgaps. However, from the viewpoint of the ambient light, the gaps betweenthe pin-mirror arrays 735 are sized to allow a significant portion ofambient light rays 760 through the glasses so as to enable an AR viewingexperience.

In use, image light rays 745 emanating from a given image pixel of theimage generating optics 750 experience or interact or reflect off atleast one pin-mirror 730 and preferably several, but each pin-mirror 730re-directs light for a multitude of pixels towards the eye box 722.Depending on the design of the combiner 705 and the AR headset 700, apin-mirror array 735 within a combiner 705 can have a total pin-mirrorarray size of 20-30 pin-mirrors 730 per eye, and maybe as many as 100total pin-mirrors 730. The pin-mirror array size can also be expressedas the total area (see FIG. 7D) of a combiner that includes pin-mirrors,which can for example span a 50 mm width and 40 mm height. Also, sincethe size of the pin mirrors 730 are rather small, the transparency ofthe glasses for ambient light can be maintained as well. However, ascompared to the conventional AR headset of FIG. 3B with diffractiongrating light output couplers, which have optical features on themicrometer or nanometer scale, these pin-mirrors are relatively large(millimeter scale) and do not cause significant diffractive effects.Image light reflected by the pin-mirrors 730 will be directed to aviewer's eye, while ambient light rays 760 from the broader environmentthat the viewer resides in, can transit the gaps between thepin-mirrors, to reach a viewer's eyes. The image light directed by thecombiner 705 towards the eye box 722 to produce a virtual image, as seenby a viewer, for any given image pixel, is meant to be “collimated” witha target vergence of 0.0 deg.±˜20 arcmin. As a result, the viewer canhave an augmented reality (AR) or mixed reality (MR) viewing experience.

A benefit of this approach is that with controlled fabrication andplacement of the pin-mirrors 730, deflection of image light towards aviewer can be optimized, largely independent of the further optimizationof the combiner 705 for allowing transit of ambient or environmentallight to a viewer. In particular, by fabricating a combiner 705 with lowarea-density, but highly reflective imbedded or internal mirrorsurfaces, whose reflectivity us independent of the inner and outercombiner surfaces, can enable a substantially transparent lens, with thesame effective reflection as the typical partially reflectivesingle-surface combiner.

Considering FIG. 7A, the efficiency of directing virtual image lighttowards a viewer's eye largely depends on the reflectivity (e.g., R˜92%)of the pin-mirrors 730, and the apparent high fill factor of both thepin-mirrors 730 and the pin-mirror arrays 735, as seen or experienced bythe incoming virtual image light. As seen by the incoming virtual imagelight, the circular or elliptical pin-mirrors have a fill factor of˜80%, as compared to being square or rectangular mirrors. The apparentfill factor from one pin-mirror array 735 to another, as experienced bythe image light, will depend on the optimized array positioning,manufacturing tolerances, and the stability or robustness of the productthereafter. Assuming an apparent array fill factor of ˜92%, and notincluding optical absorption of the slices or substrate, and Fresnellosses or AR coatings at the surfaces, the overall optical efficiency ofthe pin-mirror array can be estimated as ˜0.92*0.8*0.92≈0.68, which canbe a significant improvement over existing ARHS light guides. Bycomparison, with respect to the incident ambient light, the pin-mirrors730 are sparsely located and small, and the apparent fill factor is low(e.g., ≤20%, and preferably ≤10%). Therefore, the pin-mirrors 730 andpin-mirror arrays 735 can be optimized within a combiner 705 and a lightguide based ARHS, to improve both image quality and a light efficiencyor high fill factor for the virtual image light (745), with littleimpact on a high light efficiency or low fill factor for the ambientlight (760). Other efficiency optimizations, including for nominallyequal ambient and image light through the pin-mirror array efficiencies(e.g., ˜75% each), or for image light efficiency being higher thanambient light efficiency, can be favored, depending on the application.

The improved combiners 705, with pin-mirrors 730 also can provide someof the benefits of pin-hole glasses, for a user viewing the projectedvirtual image content. As previously discussed, pin-hole technologyreduces the blur circle by filtering out some of the light waves. Thisis similar to the effect that is realized when a person squints theireyes to improve vision. Squinting reduces the size of the de-focus lightrays that land on the retina. Pinhole glasses include a series of smallholes within an opaque visor that allows only a portion of the light topass through and enter the pupil of the eye. The pin-mirrors 730 canhave an analogous effect for viewing of the virtual image content. Theeffective size of the pin-mirrors 730 is increased by the pinhole opticseffect, which both increases the depth of field and provides a wide eyebox. The apparent resolution that can be achieved by a pin-mirror 730with a diameter of D, is roughly 1.97/D arc minutes.

In the case of the AR headset 700 of FIG. 7A-D, the image generatingoptics 750 and associated input coupler optics 755 can be located to theleft or right or the viewer's eyes, near the temples, with the imagerand associated optics positioned along the side of the viewer's head.Alternately, the input coupler and imager and associated optics can belocated above the viewer's eyes, near the forehead, so that image lightis coupled into the combiner at or near the upper edge. In this case, asshown in FIG. 8, each of the imbedded pin-mirrors 830 are generallyoriented with a vertical tilt to direct the image light towards an eye.

Aspects of the fabrication of the combiners were previously discussedwith respect to FIGS. 4-7. In greater detail, it should be understoodthat the inner surface and outer surface (i.e. respectively 727 and 728in FIG. 7A), to provide a functional light guide, whether fabricatedwith or without a substrate (i.e. substrate 440 in FIG. 4A, should benominally parallel. In the preferred design, the assembled light guideor combiner or eyepiece is flat, with only modest wedge (e.g., ≤0.1o).Also, the spatial flatness of the individual outer surfaces (i.e. 727and 728 respectively in FIG. 7A) of the light guide eyepiece orlightguide visor should be flat to roughly λ/4, which is 138 nm, for awavelength of 550 nm. A preferred thickness tolerance for exemplarylight guide combiners is ˜±0.20 mm. A 60-40 surface quality would betolerable. If a substrate is included, it should nominally be made withthe same material, whether glass or plastic, as the slices in thevarious embodiments. The exemplary visors can also be fabricated with anouter frame (not shown), similar as to that for a pair of eyeglasses, tohelp hold the slices together.

The various exemplary improved light guide combiners can also befabricated by casting or injection molding, for a relatively low cost.By comparison, a typical waveguide combiner with diffraction gratinglight couplers has a coating or texture on one or both of the inner andouter combiner surfaces, to create a partially reflective element, butthese coatings or textures also reduce the transparency of the element.This results in a tradeoff between transparency and reflectivity, andnever quite fulfills either requirement.

In an exemplary embodiment, the structure of a combiner or visor can beattached by fusing or joining two or more slices together with a mirrormesh sandwiched between them. In another embodiment, this may beachieved by creating small holes penetrating the surface of the visor toa particular depth, and then depositing mirrors within the holes. As anon-limiting example, gallium could be injected into the holes in aliquid state and once solidified, they could be used as mirrors. Itshould be appreciated that these are simply exemplary techniques thatcan be used to create embodiments of the visor or combiner and shouldnot be construed as a limitation.

It should be appreciated that while FIGS. 4A-7D may seem to suggest thatthe individual micro-mirrors and rows or linear arrays of micro-mirrorsare fabricated across a combiner on parallel planes of planar edgefacets, that those skilled in the art will understand, as depicted inFIG. 8, that the combiner 805 can provide a spatially variant tilt ofthe pin-mirrors 830 or pin-mirror arrays 835 across the combiner 805.Relative to the light input edge or input light coupler, and the innerand outer surfaces of the combiner 805, the mirror facets 825 can betilted at spatially variant angles across the light propagation lengthof the combiner. For example, near the virtual image light input end ofthe combiner 805, the pin-mirrors 830 can be tilted at a nominal angleof Ø1 (i.e. 47o as a non-limiting example), while in the center regionof the combiner 805, nearest the eye box 820, the pin-mirrors can betilted at the nominal angle of Ø2 (i.e., 45o as a non-limiting example),and at the far end of the combiner 805, furthest from the input end, thepin-mirrors can be tilted at the nominal angle of Ø3 (i.e., 43o as anon-limiting example). An optimization with a spatial tilt variation ofthe pin-mirrors 830 provides an additional degree of design freedom thatcan ease the optical design of the imaging optics, or the combinerdesign and fabrication specifications. Also, if the improved ARHS 800has an eyepiece or combiner with tilted pin-mirrors (FIG. 8) or tiltedpin-mirror arrays (see FIG. 5 and FIG. 14), and the pin-mirror coatingsare reflective dichroic notch coatings, than the pin-mirror tilt orcombiner curvature can help compensate for the spectral shifts thatoccur when dichroic coatings are tilted. Alternately, the notch positionof the dichroic coatings can be deposited to vary spatially across thecombiner so that the apparent notch spectral position to the eye boxappears constant. The widths of the pitch 832, or of the gaps betweenthe pin-mirrors 830, can also be optimized spatially across a combiner805 (e.g., optimizing the spatial frequencies). Although the size,shape, and tilt of the actual pin-mirrors 830 can be optimized to varyspatially across a combiner, a goal can be that the apparent pinholesize, as seen by a viewer, is nominally constant (e.g., within ±15% ofaverage) across the combiner. The optimization of these parameters canalso benefit optical efficiency, in terms of how much virtual contentimage light can be reflected towards the eye box, and thus also thepotential sizes of the pin-mirrors. For example, if spatial tiltoptimization allows smaller mirrors, then transmission or transparencyfor ambient light can be increased. Also, the size and positioning ofthe eye box 822 can be improved with spatial optimization of thepin-mirrors 830, to either side and to the top and bottom. The preferredrange for the size of the eye box 822 is 10-15 mm, both horizontally andvertically.

Additionally, in any of the embodiments but described with regards tothe embodiment in FIG. 8, the imbedded edge facets 825 on which thepin-mirrors 830 are fabricated, and indeed the entire combiner 805, canalso be fabricated to have a curvature, which can be concave or convex,and symmetrical or asymmetrical (e.g., cylindrical). For example, theentire light guide or combiner 805 can have a curvature, or compoundcurvature, oriented inwards towards the eye. Thus, the facets 825 of theslices 810 upon which the pin-mirrors were fabricated can also havecurvature. Alternately, the combiner 805 or light guide can be a flatdevice with nominally plane parallel surfaces, but one or more facets825 upon which the reflective mirror coatings are fabricated, can befabricated with a curvature, resulting in a curvature for a pin-mirrorarray 835. The individual pin-mirrors 830 can also have a localizedcurvature or scalloping on a facet 825 that either is otherwisenominally flat or which has its own curvature with a much larger radius.Curving the individual pin-mirrors 830 or the facets 825 can provideadditional design freedom for the entire optical design, including thatof the image generating optics 850. For example, a large radius ofcurvature (e.g., ≥150 mm), or small optical power, that is eitherconcave or convex, can be used. The pin-mirrors 830 can also have aspatially variant curvature, for example where the pin-mirrors 830proximate to the eye box 822 are flat, and the pin-mirrors 830 nearestthe sides or edges can be optimized with curvature. Curvature of thepin-mirrors 830 can be useful to help correct for spherical or chromaticaberration, or to modify or assist the collimation of image lighttowards the eye box. As an example, a design for an improved light guidetype AR headset 800 with a pin-mirror based combiner can provideaspheric or free-form lens elements that work in combination withpin-mirrors 830 that are fabricated with spatially variant tilts,widths, shapes, or curvatures.

In a design with multiple curved pin mirrors, each pin mirror cancontribute part of the imaging function of the micro-display. However,if these curved pin-mirrors don't belong to a whole large curve, thegenerated sections of the image may not be combined seamlessly anddifferent twists of the image portions can happen, meaning that theperceived image has a local or spatially variant and unintendeddistortions. One way to reduce or avoid image twist is to have eachcurved pin mirror belong to one identical or common large curved surfaceimbedded within the combiner (each pin mirror is a part of a whole largecurved mirror).

Alternate versions of an improved light guide based ARHS with imbeddedpin-mirrors 930 are shown in FIGS. 9A-E. In particular, in the versionshown in FIG. 9A, a large curved mirror 970 is provided near the bottomof the combiner or “eyepiece” 905 that can function to collimate thebeams along the horizontal plane of the eyepiece. The AR headset can beequipped with image generating optics 950 that includes an LED array 975that emits virtual image light 945, and beam shaping optics (i.e.,lenses 980) that alter the light and direct it into a light guidecombiner 905. In the exemplary embodiment that is shown, a firstcylinder lens nominally collimates the image light from pixels in theLED array 975 in the narrow (4 mm wide) direction of the light guide.FIG. 9E depicts a perspective view, showing light propagation for threefield positions. The light will reflect off an input coupling edge facet(915) and be directed into the elongate portion of the light guidecombiner 905. In this orientation, the image light will propagate inpart by TIR through the light guide 905. In the orthogonal orientation,a second cylinder lens (980) can alter the image light from beingdivergent to convergent, before the image light encounters the inputcoupling edge facet 915. A preferred configuration for this system is toprovide the LED array 975 and associated optics above the eyes, so imagelight is directed from the forehead downwards into the combiner 905.

Then, as shown in FIGS. 9A-E, a design for a visor 900 provides acombiner 905 with flat pin mirrors 930, but with a large cylindricalcurved mirror 970 near the bottom of the eyepiece/combiner 905. Once thelight has propagated through the length of the light guide combiner 905,it can hit an imbedded cylindrical curved reflector 970 and be reflectedback, and then become nominally collimated in the wide direction of thelight guide combiner 905. After collimation by the large curved mirror970 at the bottom of the combiner 905, the beams are reflected back tothe eyepiece and then can hit the array of pin mirrors 930A or 930B(collectively referred to as 930) and reflect out of the eyepiece andtowards an eye box 922 where a human eye can view a virtual image atinfinity. As an example, a light guide with a 60×50 mm size, and athickness of 4 mm, has a curvature for the large curved reflector 470 of˜30 mm, so as to modify or collimate the virtual image light for thehorizontal field of view within the eyepiece.

In the prior configuration of FIG. 7A-D, the pin-mirrors 730 were tiltedto face the incoming image light and deflect it towards the eye box 722.But in the FIG. 9A-9E configurations, the pin-mirrors 930 ormicro-mirrors can be tilted to face (e.g., at 30o) the curved mirror970, so as to re-direct light reflected from the curved mirror 970towards the eye box 922. This means that during an initial transit ofimage light through the light guide combiner 905 towards the curvedreflector 970, some image light can encounter the “back side” of thepin-mirrors 930, and be deflected outwards, towards the ambientenvironment, where this light may be noticed by other people. To reducethis effect, the pin-mirrors 930 can be fabricated with a “back side”light absorption coating (e.g., ≥97% light absorbing). Similar black orlight absorbing coatings can be provided on the light guide edges,including portions of the edge facet 915 that are not used for couplinginput virtual image light 945 into the light guide combiner 905, so asto attenuate stray light and prevent its observance by either a vieweror people in the ambient environment.

It is also noted that the curved reflector 970, rather than beingimbedded, can be provided as a mirror coating applied to a curved endface of the light guide or combiner 905. It is also noted that FIGS.9A-E depict two versions, relative to the arrangement of the pin-mirrors930 into an array. In one version (FIGS. 9A-9B), the pin-mirrors 930Aare distributed in a pin-mirror array 935A that spans most of the areaof the light guide combiner 905. In a second version (FIGS. 9C-9D), thepin-mirrors 930B are tightly clustered in a spatially variant pin-mirrorarray 935B with two adjacent groups, with a partial gap 937 betweenthem. The partial gap 937 is used prevent light loss in the centerfields and allow more light from the center field to propagate throughto the curved mirror 970 and then to pin mirrors 930.

The pin mirrors 930 are used to couple out the light reflected from thecurved reflection surface (970) in the waveguide. As an example, 2 mmwide pin-mirrors 930 tilted at 30 deg. will seem only 1 mm tall. Anoptimized pin-mirror array design can use only 20-100 pin-mirrors 930.With more pin mirrors 930 occupying a larger area, the amount of coupledout light will be larger, and thus the image brightness can increase.However, there are other trade-offs. As one example, the eye relief, orthe distance between the eye box 922 and the improved light guidecombiner 905, or eyepiece, can be reduced. The system has a finiteworking distance, as given by a distance between the curved reflector970 and its exit pupil (e.g., the location of the eye box 922). Thus,the further the pin mirrors 930 are from the curved reflector 970, thesmaller the corresponding eye relief will be. Based on this, the pinmirrors 930 cannot be too far “above” or away from the curved reflector970. Second, to expand the array area given to the pin mirrors 930, canblock more ambient light. These, and other trade-offs can be addressedduring the optimization process of the pin mirrors 930, by determiningfactors including the number, size, positions, and the ambient andvirtual image light fill factors, of the pin mirrors 930.Advantageously, this version of the improved light guide based ARHS 900with imbedded pin-mirrors 930 and curved reflector 970 can be optimizedto present virtual image light 945 to an eye box 922 over a horizontalFOV of at least 100 degrees.

In the improved light guide and pin-mirror based ARHS shown in FIGS.9A-E, the light propagates through a single light guide, to reach thecurved reflector 970, and then to reach pin-mirrors 930. Alternately,the image light could be initially coupled into a first light guide (notshown) that is nominally parallel to, but slightly offset from, by athin air gap, a second light guide having the pin-mirrors. The two lightguides merge or are contacted with an index matching material in theregion proximate to the curved reflector, so that image light can thenbe directed towards the pin-mirrors. A random arrangement of spacerbeads or posts can be used to maintain the air gap between the lightguides. While this configuration can be mechanically more complicated,virtual image light is not lost by encountering the back side of thepin-mirrors.

FIG. 10A-C depict different views of a viewer's eye receiving virtualimage light from part of an AR headset having pin-mirrors. For greatercontext, FIG. 10 depicts a 3D or isometric view of a viewer's eye 1001receiving virtual image light rays 1045 from part of an improved ARheadset 1000 having pin-mirrors 1030. Image light is coupled into theplanar inner surface of the light guide combiner 1005, and reflected offof an angled edge facet towards the pin-mirrors 1030. In particular,this image depicts part of an ARHS with a curved bottom reflector ofFIG. 9A-E.

The “screen-door door” effect is normally denoted as when the fine linesseparating pixels become visible. This can be solved by increasing theresolution of the display. In the heads-mounted display, the“screen-door” effect can occur because a single display is stretched toprovide a large field of view and the fine lines between pixels becomemore visible. As the individual pin-mirrors are both small and close tothe eye, they are unlikely to cause a significant screen door effect. Ifthe pin mirrors 1030 were larger, they would need to be further awayfrom each other so as not be seen. Additionally, there is a risk ofperceptible moiré occurring. But for this light guide ARHS, as long asdifferent layers or rows of pin mirrors 1030 do not overlap with eachother, a “moiré pattern” is unlikely to occur. Also, given therelatively large size and pitch of the pin mirrors 1030 and pin mirrorarrays 1035 (mm dimensions), compared to the size of the projected imagepixels, visible moiré between the pin-mirror arrays and the imagecontent is unlikely. Additionally, the physically positioning or pitch1032 or shape of the individual pin-mirrors 1030 in the pin-mirror arrayor sub-arrays can be randomized to reduce the risk of perceptible moiré.

In general, the introduction of pin-mirrors in the combiner providesmany additional degrees of freedom for designing the visor, the imaginggenerating optics, and the AR headset. In particular, a variety ofparameters, including pin-mirror size, shape, tilt and spatial tiltvariation, gap spacing or pitch, pin-mirror or facet curvature, andoverall combiner curvature can become available.

FIG. 11 outlines an optimization method 1100 that can be used to designthe plurality of pin-mirrors, a combiner, and an AR headset generally(including the displays of FIGS. 7A-D, FIGS. 9A-E, FIGS. 10A-C, FIGS.12-13, and FIGS. 14-15). In an initial input step 1110, values or rangesfor input parameters related to the general design of the combiner andlight guide are provided, along both relevant parameters related to theimaging optics and input light coupling optics, and the parametersrelated to illuminating a viewer's eye with image light. These systemparameters (P) can include at least the target FOV, the light guide andcombiner or visor size, the light guide thickness, the eye box size andposition, and the eye relief. In a second initial input step 1120,values or ranges for parameters specific to the pin-mirrors andpin-mirror portion of the combiner are provided. These pin-mirrorparameters (P) can include the minimum pin-mirror size (to guaranteemanufacturability, and reduce image blur), the maximum pin-mirror size(to avoid pupil focus), the maximum or nominal pin-mirror spacing (toensure optical overlap), the minimum pin-mirror array size (to ensureminimum eye box size), the maximum pin-mirror array size (to fit in aneye glass lens), the pin-mirror array shape or outer contours, and thepin-mirror coating (both reflective and light absorbing) performance.Other input parameters (P) can include the length of a pre-pin-mirrorlight guide portion, the pin-mirror array fill factors (e.g., a highfill factor for the virtual image light and a low fill factor for theambient light), the facet or pin-mirror tilt, multi-plane pin mirrorarray parameters (FIG. 5: e.g., the extent of the sub-arrays, pitchbetween sub-arrays, parallelism or relative skew or tilt between thesub-arrays, the pin-mirror positioning within the individual sub-arrays,and the avoidance of moiré), facet or pin-mirror curvature, and thespatial variation of a facet or pin-mirror tilt or curvature in either ahorizontal or vertical direction. Although the pin-mirror arrays andsub-arrays are depicted as having the pin-mirrors arranged within anominally rectangular area, the outside shape or contour of thepin-mirror arrays need not be rectangular. In particular, the pin-mirrorarray area contours can also be optimized using appropriate parametersso that the array outer edges more closely follow the edges of the eyepiece, which can be curved and shaped to better fit to the contours of aviewer's face. Using the parameters that are input in steps 1110 and1120, then initial system performance metrics can be calculated in step1130, and compared to target values.

An iterative optimization process than follows, via steps 1140 and 1145,in which values for the input parameters can be modified and newperformance values calculated and tracked. This optimization process canuse a damped least squares method, a global optimization method, orother calculative techniques. Depending on the algorithmic optimizationapproach, an additional step 1125 can be included to provide userdefined or automatic weighting values that can be used in optimizationmerit function (e.g., M=A1P1W1+A2P2W2+A3P3W3+ . . . ). The weightingfactors (W) can be applied to both the system or pin-mirror parameters(P) and the system performance metrics (A). The optimization method 1100then nominally ends at an output step 1150, which provides “final”optimized values for the various parameters, as well system performancevalues for the performance metrics. The performance metrics determinedin steps 1130 and 1145 can include image brightness, image lightefficiency, image color or intensity uniformity, image blur or imageresolution (MTF), field of view and eye box size, and ambient lighttransmission or transparency. Of course, also, the input parameters andmerit function weightings can be changed and the method re-run.

The optimization can be separated into the optimization of projectionoptics and the optimization of the size and arrangement of the pinmirrors. Optimization or design of the projection optics can becompleted by using sequential mode in Zemax or CodeV without consideringthe out-coupling of the lightguide eyepiece (pin mirrors) and can beachieved by operating the merit function. Whereas, optimization of thesize and arrangement of the pin mirrors can be completed in designsoftware by setting each pin mirror as a detector that can detect howmuch total power is reflected to be coupled out of the lightguideeyepiece and the individual power reflected corresponding to each singlefield of light. By doing this, the relationship between the size andplacement of the pin mirrors, the total reflected power, and thereflected power for each field, can be set up and evaluated, todetermine an optimal number of pin mirrors, the pin mirror size(s), andthe pin mirror placement. In practice, in the various embodiments theoptimization of the pin-mirrors and combiner can inform or limit theoptimized design of the imaging optics, and the design of the imagingoptics can inform or limit the optimization of the pin-mirrors andcombiner. In general, the preferred fill factors can vary with the ARHSdesign and fabrication, coating properties, and the expected viewerapplications.

In the prior discussions, it is generally assumed that the visors orcombiners for the left and right eyes would have identicaloptimizations, except that for any spatial variations, one combinerwould be a mirror image of the other. However, for some applicationspecific purposes, or for a customer specific design, the visors orcombiners can be optimized differently, as can the associated imagingsystems.

FIG. 14 depicts an alternate embodiment for an improved light guidebased AR headset 1400 having a combiner 1405 with an array ofpin-mirrors 1430. In particular, FIG. 14 depicts a line scanning ARdisplay system that can be used for left eye or right eye viewing, inwhich an image source 1440 (e.g., a micro-LED array) provides imagelight 1445, via collimation and projection optics 1410, a scan mirror1420, through a lightguide or combiner 1405 having pin-mirrors 1430arranged on a plurality of pin-mirror sub-arrays 1435, to an eye 1460 atan eye box 1450. Optics 1410 can be refraction, diffraction, reflection,or electrical-controlled diffraction based, or combinations thereof. Thevisor or lightguide combiner 1405 can also be shaped and contoured toimprove the fit to a viewer's face.

It is noted that at present, it can be difficult to fabricate and sourcesmall, bright 2D micro-LED arrays 1440 with tightly packed addressableRGB image pixels (1542 see FIG. 15). As an alternative, a tri-linear RGBLED array light source can be used. For example, the LED array sourcecan be a true 1D tri-linear array that provides a line of addressableLED pixels having 1×4096 red light emitting pixels, parallel to asimilar respective rows of green light and blue light emitting pixels.Alternately, as shown in FIG. 15, the image source 1540 can be a devicethat can be described as a 2D micro-LED array or block-width tri-linearmicro-LED array. In particular, FIG. 15 depicts a portion of an LEDarray device with an arrangement of LED pixels as three linear areas orblocks such that a parallel linear array of Red (R) pixels 1542 isadjacent to a parallel linear array of Green (G) pixels 1542, that isadjacent to a parallel linear array of Blue (B) pixels 1542. Forexample, each block or linear array of pixels, whether R, G, or B, cancomprise 50×8000 pixels. The LED emitters in a given line (e.g., 50pixels wide) are individually addressed and controlled, and at any pointin time, during scanning and image display, they can be providing anintensity of image light for different details of the displayed AR imagecontent. This second approach, with a block-width tri-linear micro-LEDarray, enables embodiments of the ARHS to provide a brighter image.

Within a linear micro-LED array light source 1540, individual lightemitting pixels 1542 can also be square or rectangular in aspect ratio.As an example, an individual light emitting pixel, whether R, G, or Bcan have a nominal side dimensions of 2.5-5.0 microns width, althoughthe pixels can be smaller (e.g., 1.0 microns wide) or larger. Each blockor linear array of pixels, whether R, G, or B, can comprise 8000×50pixels. Thus, for example, with 3.2 micron square pixels, each of therespective color arrays would be 160 microns wide, and 25.6 mm long, toprovide an overall linear type device or image source 1540 that is ˜0.5mm wide and 25.6 mm long. The linear arrays of RGB image pixels 1542 inFIG. 15 can be provided with other arrangements of the colors, such R,B, G, and the number and size of image pixels need not be identical fromone color array to another. The LED array can also be equipped withmicro-optics, such as a lenslet array (not shown), to help with beamshaping. For example, a custom designed micro-lens array, aligned andmounted to collect and redirect light from the LED pixels, can havelenslets with customized shapes, or optical designs that are spatiallyvariant across the LED array or by color (R, G, B). Although FIG. 15depicts the tri-linear LED Array (1540) as a straight linear RGB device,the device can also be a white light, or monochrome or single-colordevice, or be curved (along an arc) or shaped. Curving or shaping thearray can better match an eyepiece (combiner 1505) in a way that is moreconformal to the human facial structure, and increase apparent lightefficiency to a viewer.

In either case, a tri-linear micro-LED array 1540 with LED pixels 1542can be used as an image source 1440 for the improved AR headset 1400 ofFIG. 14. The emitted image light is shaped by collimation optics (1410)and directed onto a 1D scanning micro-mirror 1420, through projectionoptics (1410), and into a combiner 1405 or eyepiece, to then transit thecombiner and be directed to the eye box. As shown, this combiner hasmultiple sub-arrays (1435) of pin-mirrors 1430. This system can providehigh brightness AR images to a viewer simultaneously along with thepresence of high brightness ambient light 1465. The 1D, 2D or customizedscanning system could be provided using a variety of mechanisms,devices, materials, or modulation components, including but not limitedto, MEMS devices, solid state displays, spatial light modulators (e.g.,back illuminated liquid crystal (LC) devices), modulation crystals, orbeam deflectors.

Operationally, the individual R, G, or B LED pixels 1542 can provideemitted light with 8-10 bits of modulation depth, at a display frequencyof 30-120 Hz, depending on the application and specifications. Both themodulation bit depth and display frequency can be increased (e.g., to12-14 bits, and 144-200 Hz, respectively) depending on the availabletechnologies and the value to the ARHS product. This modulated imagelight 1445 is then directed through optics 1410 to a linear scan mirror1420, which can be driven by a controller (not shown). The scan mirror1420 can be either a resonant or non-resonant scanner, with its scanoperation calibrated by a line scan position monitoring sensor (notshown). FIG. 14 depicts two tilt positions for this scan mirror, withopposite tilts. Scan mirror 1420 can be a MEMs (microelectromechanicalsystems) device, for example that is a single mirror with an activemirror 2.5 mm wide and 6 mm long, where the mirror tilts by ±7-10degrees about the width direction. Improved or optimized devices witheither smaller or larger (e.g., ±12o) scan angles can also be used. Theoptical scan range (angle) is 2× the mechanical scan range (angle). Thescan mirror 1420, which can also be designed as a linear array ofmultiple mirrors, can be provided by vendors such as PreciseleyMicrotechnology Corp. (Edmonton AB, CA) or Fraunhofer IPMS (Dresden,DE). Scan mirror 1420 can also be enabled by other technologies, such asa piezoelectric device (e.g., using PLZT) or a galvanometer. As the scanmirror 1420 tilts, the image light 1445 is swept through the light guidecombiner 1405, to reflect light off of pin-mirrors 1430, and directlight to an eye box 1450. Image light 1445 can be provided by the LEDpixels 1442, in synchronization with the scan mirror 1420 tilt, suchthat image light 1445 is directed into the eye box 1450 for an extendedduration per sweep. As image content can be provided for both directionsof scan mirror tilting, the effective operational scanning duty cyclecan be high (e.g., ˜90%).

A preferred configuration for this system is to provide the image source1440, associated optics, and scan mirror 1420, at the top, above theeyes, so image light 1445 is directed from the forehead downwards intothe combiner 1405. As previously described, a variety of pin-mirrorparameters, such as a maximum and minimum size, a pitch or gap betweenthem, and target fill factors can be defined. Then, during optimization,with an optimization method 1100 (FIG. 11), the pitch, size, shape,curvature, tilt, positioning, fill-factors, coatings, and otherparameters related the pin-mirrors 1430 and the pin-mirror sub-arrays1435, including the sub-array pitch 1432, within the combiner 1405 canbe optimized. As an example, the 1D scanning AR display system 1400 ofFIG. 14 can use an array of pin-mirrors 1430 in which the pin-mirrorshave ˜0.4-1.2 mm widths, and are spaced apart from one another by aspatially variant pitch (1432) in the ˜2-5 mm range, and combiner 1405can have a total of 300-1000 pin-mirrors 1430 distributed across one ormore imbedded pin-mirror sub-arrays 1435. But depending on the designoptimization of the pin-mirror based combiner or eye piece 1405, thenumber of pin-mirrors can be ≤50, or ≥2000, or somewhere in between. Theoptimization (e.g., FIG. 11) of the configurations of the individualpin-mirrors in the various embodiments and the pin-mirror sub-arrays inthe various embodiments, relative to pin-mirror design parameters suchas number, size, pitch, curvature, and coatings, and system parameterssuch as the target headset FOV (e.g., a WFOV ≥90o per eye), can bemotivated by many factors or performance metrics, including the lack ofvisible moiré, the apparent headset transparency for the ambient light,and the apparent brightness for display expected light. Otheroptimization or performance metrics can include factors that arespecific to a given viewer application or to the manufacturability ofthe pin-mirrors and pin-mirror arrays. The FIG. 11 pin-mirroroptimization method can also be a subset of a larger optimization methodthat includes the design of the entire combiner, or the entire ARheadset, including the design of the imaging optics, housings, andvarious light trapping or light absorbing features.

As shown in FIG. 14, the pin-mirror based combiner 1405 used in theimproved scanning and light guide based AR headset 1400, which can bestraight or curved, can be of the type with multiple planes of parallelsub-arrays of pin-mirrors 1430 (see also FIG. 5). The combiner can havecurvature or shaping to help conform to the shape of a viewer's face,and curvature can be provided only outside the area used for imagedisplay, or it can extend to within the viewed area. The AR headset 1400of FIG. 14 can also be provided with pin-mirror based combiners that areof the type with a single laterally spread pin-mirror array using asingle light guide (FIG. 7), or of the type (FIGS. 9A-E) with dualparallel light guides and a curved reflector (970) at the bottom of theeyepiece, opposite the top side image source.

The 1D scanning, pin-mirror based, AR headset 1400 of FIG. 14 also canbe advantageously adjusted for variations in interpupillary distance(IPD) amongst viewers. As an example, the device can be designed so thatnominally only 6000 pixels of an available 8000 pixels of an imagesource array (1440) are used at a given time. But the stripe of usedpixels can be selected to shift the images provided by the left eye andright eye scanning displays, to the left or right, so as to adjust fordifferent people's interpupillary distance. This capability can beenabled by a calibration set-up process or with eye tracking.

FIG. 16 depicts a portion of a trilinear display used in a 1D scanning,pin-mirror based, AR headset 1400 of the type of FIG. 14. This improvedversion provides a method to enhance brightness without requiringbrighter LEDs or stronger LED drive currents. A modified trilineardisplay 2501 is composed of several arrays or blocks 2502, 2503, and2504 of LED emitters or emitting pixels, red, green, and bluerespectively, each with multiple rows. As also in FIG. 14, the trilineardisplay of FIG. 16 is scanned with a continuously-moving one-axisscanning system 2505 to create a virtual area display 2506. Electricaldrive signals are applied to each of the LEDs such that each row of LEDscombines constructively to increase the brightness of each display rowas viewed by the user. Because of the redundancy of emitters in thecolumn or cross-scan direction, which can have 50-100 emitters per rowper color, the presence of several dead pixels per column and color canbe tolerated with no loss of image quality, and only a slight loss inbrightness. This can greatly increase the ability to tolerate higherpixel defect rate and reduce display costs, especially for challengingdisplay technologies such as mass-transferred micro-inorganic-LEDdisplays. Additionally, the tri-linear scanning architecture of FIG. 14and FIG. 16 can provide a boost in the dynamic range or modulation bitdepth provided by the ARHS by using time domain integration (TDI). Asthe driver electronics can address individual LED emitters in theindividual columns of the trilinear display 2501, the brightness of eachlight emitting pixel can be controlled, between minimum and maximumlight emitting levels, and the effective brightness of a pixel of imagecontent displayed within a frame time can be controlled over a widerrange.

According to an alternate embodiment of the present invention, theimbedded reflector array of pin-mirrors 1230 can also be used within animproved eyepiece or reflective combiner 1205 for a projection type ARglasses display (1200), but improved over the example depicted in FIG.3A. As shown in FIG. 12, instead of having a complicated partiallyreflective coating, the inner surface 1215 of the combiner 1205 willhave an AR coating so that the virtual image light 1245 penetrates intothe combiner 1205 and interacts with pin-mirrors 1230. As shown in FIG.13, a combiner 1305 can be fabricated with a combiner substrate 1310having a plurality of pin-mirrors 1330 with a spatially varying tilt.FIGS. 12 and 13 depict a cross sectional view of a 1D row or array ofpin mirrors 1230 and 1330 respectively, arranged to provide a horizontalspatial variance of tilt alignment. More completely, a 2D array oftilted pin-mirrors 1330 is provided both horizontally and verticallyover most the height and width of a combiner lens. The combiner 1305 canbe manufactured as a flat optic (FIG. 13) using a polymer or a glassmaterial, and then slumped to conform to a curvature, or complexcurvature. Alternately, the combiner 1305 can be cast or molded with thepin-mirrors 1330 imbedded within it. Once the combiner 1305 matches thedesired shape, it can be AR coated on both the inner surface 1315 andouter surface 1320.

As another option, the combiner 1305 of FIG. 13 can also be fabricatedas a Mangin mirror. Mangin mirrors are catadioptric reflectors that aremost commonly used in telescopes or printing systems. Typically, aMangin mirror's construction consists of a concave (negative meniscus)lens made of a crown glass with spherical surfaces of different radii,and with the reflective coating on the shallower rear surface. Thespherical aberration normally produced by a simple spherical mirrorsurface is canceled out by the opposite spherical aberration produced bythe light traveling through the negative lens. In the case of the eyepiece or combiner 1205 and 1305 of FIGS. 12 and 13, the imbeddedpin-mirrors 1330 can be fabricated along a curved inner plane that has ashallower curvature than does the inner surface 1315, so as to providethe reduced spherical aberration benefits. This improvement can, inturn, ease the image quality requirements imposed on the optical designof the projection optics within the imaging systems 1340.

A completed combiner can then be used as combiner 1205 in the improvedprojection type AR glasses 1200 of FIG. 12, in which it can be used toredirect virtual image light that is incident on the combiner at theinner surface 1215, off the imbedded pin-mirrors 1230 and towards an eyebox 1250. Because of the pre-fabricated tilt variation to thepin-mirrors 1230, less curvature can be required of the combiner 1205,or less severe divergent beam angles from the imaging system 1240, orboth. Thus, the combiner and/or imaging system can be easier to designinto the glasses or provide better performance. Also, this illustratedembodiment also advantageously reduces the blur circle, as size of pinmirrors 1230 within the combiner 1205 can be sized to optimize theamount of light coming from the virtual image generators or imagingsystems 1240 and limit the local FOV of that light. This operates toreduce the blur circle and make the image more focused on the retina. Aswith the system of FIG. 3A, the imaging system 1240 can use an LED witha 2D array of light emitting pixels, and a system of beam shapingoptics.

The design of the combiner 1205, for an improved projection type ARHS,can involve parameters including the coatings, size, shape, curvature,pitch or spatial frequency, or tilt of the pin-mirrors 1230, and can beoptimized using a design process similar to that of FIG. 11, althoughthe range of tilts used for the pin-mirrors 1230 can be much greater.For example, the pin-mirrors 1230 most proximate, or across from the eyebox 1250, can be arranged nominally parallel to the inner and outersurfaces of the combiner 1205. Whereas, the pin-mirrors 1230 closer to aviewer's nose can have a local tilt relative to local curved surface ofthe combiner 1205 of only 5-10 degrees. Whereas, the pin-mirrors 1230furthest from a viewer's nose, or closest to the temples, can have alocal tilt relative to local curved surface of the combiner 705 of 15-30degrees, but of opposite orientation or sign to the pin-mirror tilt usednear the nose. Although FIGS. 12 and 13 depict combiners with spatiallyvariant pin mirror tilts in the horizontal direction, the spatialvariation can be provided in the vertical direction, or simultaneouslyin both the horizontal and vertical directions. The actual designedspatially variant angles used for the pin-mirrors depends on thedesigned radius of curvature for the combiner. This curvature is againlikely to have a complex or compound shape, but a design goal can be toreduce the radius of curvature to about half of what is was without thebenefit of the pin-mirrors while providing a WFOV. For example, themaximum radius for the compound curvature of the combiner 1205 can bereduced to ˜20-40 mm, as compared to the 60 mm referenced previously.Use of spatially variant tilt of the pin-mirrors 1230 across thecombiner also provides greater freedom to optimize the optical designproximate of the combiner or the imaging optics on the nose bridge sidedifferently than on the temple side. As another example, a design for animproved projection type AR headset 1200 with a pin-mirror basedcombiner can provide both aspheric lens elements working in combinationwith pin-mirrors 1230 fabricated with spatially variant tilts orcurvatures or shapes.

The pin-mirrors 1230 or micro-mirrors preferably have a circular orelliptical shape. Alternately, the shape or pitch or fill factor of thepin-mirrors 1230 can also vary spatially across the improved reflectivecombiner 1205. For example, near the combiner, proximate to the eye box1250, the pin-mirrors 1230 can have a spherical shape, while towards thetemples and the nose, the pin-mirrors 1230 can have an elliptical shape.The pin-mirror shape can be optimized towards satisfying a goal that theapparent pin-mirror size, as seen by a viewer, is nominally constant(e.g., within ±15% of average) across the combiner. Pin-mirrors 1230 arenominally provided with a reflectivity of 85-98%, depending on thecoating materials used, and the angle of incidence of the virtual imagelight. The pin-mirror coatings can also be dichroic notch coatings, anda spatially variant tilt of the pin-mirrors, relative to the eye-box(see FIGS. 12-13) can compensate for the dichroic coating spectralshifts that typically occur with varying incidence angle. Thepin-mirrors 1230 can also be optimized for a nominal fill factor of 50%,to allow about equal fields and amounts of ARHS image light and ambientlight to reach the eye box 1250.

Depending on the design optimization of the combiner 1205 and theoverall AR glasses 1200, the angular spread of the image ray fanprovided by an imaging system 1240, as incident to the combiner 1205,can also be eased to span a lesser angular extent than previously, allthe while at least maintaining a target WFOV of ≥90 deg. This eases theoptical design requirements imposed on the imaging systems 1240,enabling these systems to have improved performance, or smaller size, orincreased clearance for the transiting virtual image light relative to aviewer's face and head.

As an alternate example of optimization, an inner surface illuminatedARHS with a pin-mirror based combiner 1205 of the type of FIG. 12 can bedesigned with a flat or nearly flat (e.g., radius ≥200 mm) combiner,imbedded pin-mirrors, and a smaller FOV (e.g., ≤50o). As anotheralternative, an exemplary combiner can be fabricated with pin-mirrorsprovided on or near the inner surface, or on or near the outer surface.In the latter case, with the pin-mirrors proximate to the outer surface,the combiner thickness can vary spatially to provide the Mangin mirrorreduced aberration benefits. As yet another alternative, the imagingoptics can use an LED array with a 1D tri-linear array, or a veryrectangular array (aspect ratio ≥10:1) of light emitting pixels, pairedwith a linear or rectangular mirror or array of addressablemicro-mirrors to provide a virtual image light with a linear scanningconfiguration. These various projection ARHS designs using a pluralityof optimized pin-mirrors or pin-mirror arrays, can be enabled usingvariants of the optimization method of FIG. 11. By comparison to theoptimization method 1100 used for light guided ARHS previously outlinedwith respect to FIG. 11, the optimization method for the visor orcombiner for a projection type ARHS may not need all of the sameoptimization parameters or metrics, but it can need or emphasize othersinstead.

Thus, the various embodiments of the invention advantageously operate tofilter the light from the virtual images and real images. In turn, thisreduces the blur circle for both sources and results in the virtualimages and the real-world images being more in focus regardless of thedepth of the real-world elements. Thus, the foreground and thebackground elements in the real world remain in focus along with thevirtual images. The pin mirrors operate as pin holes for the light raysfrom the virtual image generators and, they also operate to createactual pin holes for the light rays from the real-world environment. Inuse, each pin-mirror re-directs image light for a portion of the fieldof view that includes many image pixels. Also, the optical components inthese improved pin-mirror based ARHS systems, including image sourceoptics, projection optics, and eyepiece optics, can include optics orcomponents that can include, but are not limited to, optics that arerefractive, diffractive, free-form, or Kinoform, fresnel, combinedelements, holographic elements, metasurface or sub-wavelength structuredelements, gradient index elements, optomechanical components, spatiallight modulators, variable shape membranes, liquid lenses, differentdisplay components, or static or electrical controlled crystalmaterials.

FIG. 17 depicts a magnified view of light propagation into a singlepixel of an image from a pin-mirror based combiner 1710, with respect tothe viewer's pupil position inside an eyebox 1720. FIG. 17 also depictsa profile of a spatial variation of brightness or light intensity withinthe pixel. It can be seen that because different eyebox positions 1701,1702 and 1703 are lit by different sets of pin-mirrors 1704, 1705, and1706, respectively, the amount of light the viewer sees from this pixelcan change as the viewer's eye moves within the eyebox. This can lead toundesirable “shimmer” artifacts that are perceived by a viewer as theviewer's eye moves.

FIG. 18 depicts another mechanism that can produce a variation in pixelintensity across a virtual image. In this example, light from a displaythat is being provided for different pixels 1801, 1802, and 1803 isdirected through a light guide type pin-mirror combiner 1810 takesdifferent optical paths 1804, 1805, and 1806 within the combiner, andencounter different sets of pin-mirrors 1820, respectively, in transittowards a portion of an eyebox 1820. Because of the different lightpropagation paths, the overall out-coupling efficiency for these pixelscan be different. This light shading variation can be corrected insoftware by normalizing all pixel intensities to a dimmest one 1807;however, this inefficiently uses available light from the image displaydevice (e.g., an LED array).

FIG. 19 details an optical path that display light for a single pixelcan take in a pin-mirror based lightguide-type combiner of the type ofFIGS. 9A-E. In this embodiment, the image light 1901 entering theeyepiece 1905 through the incoupling structure 1902 travels in theeyepiece 1905 for some distance before striking a curved mirror 1903;becoming nominally collimated light 1904, and being directed through theeyepiece towards pin-mirrors 1905, 1906, and 1907. This transiting lightis redirected by the pin-mirrors 1905, 1906, and 1907, to be coupled outof the combiner 1905 and into an eyebox so as to create a virtual imagethat can be perceived by a viewer. It can be seen that if the imagelight strikes any pin-mirrors 1908 before hitting the final collimatingsurface 1903, a portion of this light can be absorbed or reflectedwithout creating an image. This reduces the optical efficiency of thesystem and contributes to image non-uniformities. In addition, in thisembodiment the image light comes to an internal focus 1909 before beingcollimated. If any pin-mirrors obstruct this internal focus, theefficiency loss can be severe, up to 100%, which can result in a “blackspot” in the image.

As can be seen, a variety of issues can occur as transiting image lighttraverses a pin-mirror based combiner, including light intensityvariations that are localized or span a wider field of view, and whichcause shimmer, shading, black patches or other artifacts that can beperceived by a viewer. As discussed previously, the pin-mirror combineror eyepiece optimization method of FIG. 11 can be used in combinationwith prudent display and optics design, to improve ARHS performance,including to reduce or avoid artifacts including these ones. Forexample, an optimization technique, such as damped least squares, agenetic algorithm, or a deep neural network can be combined with aweighted merit function corresponding to the quality of the system togenerate an optimal distribution of pin-mirrors. For example, the size,shape, and position of the pin-mirrors can be optimized to increase thetransparency of the combiner or to reduce the image non-uniformity tothe eyebox. As another example, the curvature and tilts of thepin-mirrors can be optimized together with the projection optics toincrease image sharpness.

A further optimization to image uniformity and optical efficiency can bemade by removing the constraint that the reflective elements orpin-mirrors be round or ellipsoidal “pinholes”. FIG. 22 depicts aconfiguration in which a pin-mirror combiner 2200 has reflectiveelements 2201, 2202, 2203 that are designed or fabricated to havearbitrary patterns or shapes on one or more of the sub-arrays, facets,or eyepiece layers. These variations in the pin-mirror configurationscan result from a pin-mirror design or optimization process thatincludes a randomization of the individual pin-mirrors within apin-mirror array or sub-arrays relative to pin-mirror positioning orpitch, size, or shape.

Pin-mirror combiner optimization (FIG. 11) can also allow removal of anellipsoidal constraint on the pin-mirror shape parameter so as toincrease performance, but this approach exposes several additionalissues which then must be accounted for in the optimization. Aspreviously mentioned, the present invention relies on the depth-of-fieldenhancing effect of the pin-mirrors to reduce vergence-accommodationmismatch. With an arbitrary pin-mirror pattern, care must be taken toensure this remains the case. Additionally, small features (2204) in thepin-mirror pattern, as suggested in FIG. 22, can cause undesirablediffraction in the virtual image, reducing its sharpness. To avoiddiffraction effects, small pin-mirror features or pin-mirror diametersshould ≥20 micrometers, and preferably ≥50 micrometers. On the otherhand, pin-mirrors with overly large features (2205) can obstruct theviewer's view of the outside world. Thus, optimization of the pin-mirrordesign can be limited by design constraints or ranges so as to benefitsystem performance metrics.

As discussed previously, in some embodiments of the invention (e.g.,FIG. 14), the pin-mirrors are distributed on a small (5-15) number ofdiscrete layers within the lightguide. FIG. 20 depicts the interactionof the light 2001 from a single pixel with the sub-array layers of apin-mirror light guide combiner 2010. It can be seen that regardless ofthe distribution of mirrors on the layers, the footprint of lightcoupled out of the combiner 2010 into an eyebox 2002 cannot be greaterthan the projections of the layers into the eyebox 2002. Furthermore,because of limitations in the numerical aperture of the projectionoptics 2003 and the size of the incoupling structure 2004, the lightfrom this pixel does not fill the entire combiner. Therefore, thefootprint of light coupled into the eyebox 2002 are at most equal to theprojections 2005, 2006, and 2007 of the footprints of the pixel light onthe layers.

FIG. 21 depicts these footprints 2101, 2102, 2103 in an eyebox, alongwith a human eye pupil depicted by the circle 2104. It can be seen thatin order for this pixel to be visible from everywhere within the eyebox,the pupil has to overlap with at least one of the footprints fromeverywhere in the eyebox. In this example, this is not the case; at theposition depicted by 2105, a viewer's pupil could receive no imagelight.

Knowing these issues, the combiner design optimization method of FIG. 11can be devised to compute an optimal positioning of the layers orsub-arrays to avoid underfilling or overfilling the intended eyebox.Additionally, this optimization method can allow design parameters to beconstrained to meet additional requirements; for example, a minimumspacing or pitch between any two sub-array layers or facets can beconstrained to meet manufacturing requirements.

A additional approach to the design improvement or optimization of apin-mirror light guide combiner or ARHS system, can be made by notingthat portions of the field-of-view above the viewer's eye can only begenerated by reflectors above the viewer's eye, while portions of thefield-of-view to the left of a viewer's eye can only be generated byreflectors to the left of the viewer's eye. This is illustrated in FIG.23, in which combiner combiner portions 2301, 2302, 2303, 2304 ofcombiner 2300 direct image light toward eyebox 2320 from differentdirections. Therefore, it can be preferable to design or optimize thesub-array layers or facets to be spatially variant, for example withcurvatures between segments of the combiner eyepiece layer beingdiscontinuous. This technique need not be limited to the four segmentsdepicted in FIG. 23, and with appropriately designed projection opticsmore segments are possible. With proper design, this approach can beachieved with little or no impact on the perceived image quality.

FIG. 24 depicts the most general form of a pin-mirror-type geometriclightguide combiner 2400. In this embodiment, the lightguide is atransparent structure 2401 with refractive index different from that ofair and dimensions much larger than the wavelength of light. Light iscoupled into the lightguide 2400 through an incoupling structure 2402(for example, a prism or a grating) and the light propagates through thestructure via total internal reflection, with the light from a nonzeronumber of pixels encountering at least one reflection 2403 off the outersurface of the lightguide. Internally distributed in the lightguide arestructures 2404 much larger than the wavelength of light which act asmicro-reflectors or pin-mirrors 2410 at the wavelengths generated by theimage source. The pin-mirrors are distributed three dimensionallythroughout a significant portion of the combiner. As the propagatinglight strikes these reflectors, it generates rays 2405 which are coupledout to a viewer's eye. Importantly, these reflectors need not beparallel to each other, nor flat; what is important is that all of thelight from a single pixel which couples into a viewer's eye forms a setof nominally collimated rays. A necessary condition on the curvatures ofthe structure 2401 for augmented reality systems is that it does notaffect the direction of ambient rays of light 2406 passing through it,so as to not distort the real world; however, this restriction need notapply for virtual or mixed reality systems.

In some embodiments of the invention, the reflective structures 2404could be electrically controllable optoelectronic devices (for example,liquid crystals or electro-optic materials), which can be dynamicallymodulated to enhance the image sharpness, uniformity, or combinertransparency based on content, gaze, or surroundings.

In order to generate the pin-mirror based light guide combiner depictedin FIG. 24, an optimization method (FIG. 11) can be applied whichincludes metrics for image quality, transparency, shimmer, uniformity,and depth-of-field to compute the best result. This method can take theform of a single algorithm, or can take the form of several individualsteps to optimize different sets of controlling parameters. Byincreasing the degrees of freedom available to the optimization routinethrough removing geometric constraints, performance metrics of thepin-mirror combiner and the ARHS system can further be improved. Some ofthese same additional optimization parameters and constraints can beapplied to the design optimization of the pin-mirrors and otherstructures in the pin-mirror based combiners for projection type ARHS(e.g., FIG. 12).

In order to manufacture such a combiner as depicted in FIG. 24, thelightguide could be fabricated via molding, casting, or grinding fromglass or a photosensitive polymer. The reflectors could be multilayerdielectric mirrors (for example, distributed Bragg reflectors) writteninto the lightguide using femtosecond glass densification or two-photonexposure of the photopolymer to create structures smaller than thewavelength of light.

In some embodiments of the invention, such as the 1D scanning,pin-mirror based, AR headset 1400 of FIG. 14, the image that isgenerated by scanning system benefits from a persistence-of-visioneffect of the human visual system, to electronically display at lowerresolution while creating a virtual, high-resolution display. One ofthese embodiments uses a trilinear device, such as an LED display (FIG.15 and FIG. 16) with three rows with different emission wavelengths anda single-axis scanning mirror to generate a high-resolution, full-colordisplay.

As noted previously, in a head-mounted display, it is desirable for thedisplay to be very high-resolution, especially if the field-of-view islarge, in order to increase the angular resolution of the virtual image.However, for viewer comfort, it is also desirable for components andoverall headset to be small. These two constraints together require adisplay with a small pixel pitch. However, because the areal emissionintensity of display materials are limited by thermal and otherconsiderations, the light output of such a small pixel can be low. Yet,for augmented reality systems, high luminous output is desired tomaintain sufficient contrast in bright (for example, daylight) ambientconditions. With respect to resolution, a goal is that the MTF does notfall below thresholds related to the display pixel pitch (5-15 microns)and the eyebox size (e.g., 10 mm wide horizontally and vertically).

But it can be challenging to construct micro-displays with very smallpixel pitches between the light emitting pixels. For example, AlGaInPred inorganic LED array devices have progressively lower quantumefficiencies as the light emitter size shrinks. FIG. 25 depicts aportion of a modified multi-row tri-linear display 2601, of the typeused in a 1D scanning, pin-mirror based, AR headset 1400 of the type ofFIG. 14 or FIG. 16. In this example, adjacent rows 2602, 2603, 2604, and2605 of light emitting pixels are alternately offset by a fractionalpixel pitch (for example, ½, ⅓, or ¼ of a pixel). Although the materialsets can vary by color, this approach can be applied to the LED emitterarrays in each color, R, G, B. This approach can also increase thespatial resolution of the ARHS display by multiplying the apparentresolution by 2, 3, or 4, respectively.

It can be seen that the increased resolution due to the tri-lineardisplay layout in FIG. 26 is not exact; rather, if the pixels are equalin dimension to the pixel pitch, the resulting image is equal to thetrue image convolved with a 2×2, 3×3, or 4×4 box filter. The apparentimage resolution can be further increased as in FIG. 26, where alithographically patterned opaque mask is 2701 used to reduce theeffective fill-factor of the display 2702. Alternatively, in FIG. 27 anengineered micro-lens 2801 array having a fill-factor under 100% is usedto constrain light emitted by the light emitters 2805 in the display2802.

Another method to increase the apparent resolution of a tri-lineardisplay, such as the device of FIG. 25, is to compensate for thebox-filtering effect (described above with respect to FIG. 26) byreplacing the software anti-aliasing filter with a deconvolution of amathematical model of the box-filter and the original anti-aliasingfilter. In combination, the software anti-aliasing filter is deconvolvedagainst the display box filter to create a modified anti-aliasingfilter. The resulting image produced by the modified anti-aliasingfilter plus the pixel box-filtering effect will appear to have beengenerated from a higher resolution LED array than was the originalanti-aliasing filter. It is noted that the ARHS can be equipped withsoftware that provides other image enhancement or compensationfunctions, including for distortion, blur, lateral color, and color orbrightness shading.

It can be seen by persons skilled in the art that the above trilineardisplay can be used in conjunction with any projection or combinertechnology. FIG. 33 depicts how a trilinear 3301 display in conjunctionwith a scanning system 3303 can be used to project a magnified image3304 through the use of cylindrical projection optics 3302, in the sameway that an area display 3305 can be used with circularly symmetricprojection optics 3306 to project a magnified image 3307. The designapproach (a) using the trilinear display is advantageous because itallows for the generation of very large, high-resolution virtual areadisplays from smaller physical displays, needed when a high-resolutionphysical display is commercially or technologically infeasible. It alsoallows the generation of full-color displays with no color samplingartifacts using technologies where striped color displays may beinfeasible. Additionally, it increases the defect tolerance on thedisplays by allowing for several defects per column, rather than severaldefects over the entire extent of the display. The trilinear displayalso decouples the linear dimension of the display from the number ofpixels on the display, granting another degree of freedom in increasingthe sharpness, reducing the cost, or reducing the size of the projectionsystem in situations where the number of pixels is constrained by costor brightness considerations. In addition, the cylindrical optics 3302can be smaller than the circularly symmetric optics 3306 due to reducedor eliminated optical power in one axis.

FIG. 34 depicts an example of an ARHS optical system utilizing atrilinear scanning system without a pin-mirror based combiner. In thissystem, trilinear microdisplay 3401 is relayed by the projection optics3402 into the scanning system 3403. 3403 scans an image which isprojected onto the free-space, semi-silvered reflector 3404 to relay amagnified image into the wearer's eye 3405. In some embodiments, 3404could utilize narrow-band dielectric coatings to increase transparency.This system can provide substantial size advantages over conventionalfree-space systems utilizing a large display in place of the projectionsystem consisting of 3401, 3402, and 3403. By enabling the use ofmicro-inorganic-displays it also provides substantial brightnessimprovements over relay projection systems utilizing a micro-organic-LEDdisplay, and improved contrast over systems utilizing aliquid-crystal-on-silicon (LCOS) display.

FIG. 35A and FIG. 35B (collectively FIG. 35) depicts an example of anARHS optical system utilizing a trilinear scanning system in conjunctionwith a free-form prism combiner. Trilinear display 3501 is relayed bythe projection optics 3502 into the scanning system 3503. Scanner 3503projects a virtual display, the light from which is coupled into thefree-form prism combiner 3504. Combiner 3504 contains one or morepartially reflecting surfaces which serve to overlay the virtual imageon the real world. This system has substantial advantages overconventional free-form prism designs by enabling the use ofmicro-inorganic-LED displays, which have higher brightness and longerlifetime than micro-organic-LED displays and higher contrast thanliquid-crystal-on-silicon displays. It also enables a largerfield-of-view in the direction of the scan, by reducing or eliminatingthe need for optically powered surfaces in that direction.

FIG. 36 depicts an example of an ARHS optical system utilizing atrilinear scanning system in conjunction with a holographic waveguidecombiner. Trilinear display 3601 is relayed by the projection optics3602 into the scanning system 3603. Scanner 3603 projects collimatedlight (corresponding to a virtual display at infinity) into theincoupling grating 3605 of a holographic combiner 3604. In someembodiments, combiner 3604 could consist of multiple stacked combinersin order to compensate for the chromatism of a holographic element. Thelight is guided by the waveguide section 3606 of the combiner 3604,encountering one or more outcoupling areas 3607 which couple the imagelight into the viewer's eye 3608. By utilizing a trilinear display withmultiple rows this design can compensate for the relatively lowefficiency of holographic combiners. By using a trilinear display togenerate one axis of the field of view optically rather than throughscanning, the resolution of the system can be improved—projectionsystems utilizing a single flying spot are typically limited to 7resolutions of under 2 million pixels due to considerations surroundingthe scanning system. By using an incoherent light source such as an LEDdisplay over a coherent light source such as a laser, speckle artifactsarising from self-interference can be avoided.

As another variation, the tri-linear light source display, whether in anARHS using a pin-mirror based combiner, or otherwise, can instead be aquad-linear type system. In particular, the tri-linear light emitterarray or pixelated display of FIG. 15 or FIG. 16 can have a fourth blockof light sources (not shown). As an example, this device can have afourth block, with some quantity of rows and columns of LEDs, althoughthe number of columns may be different than is used in the R, G, Bblocks. This fourth block could emit another color of light, such asorange or an orange-yellow color, and be provided to expand the colorgamut or the portion of color space supported by the display. As anotheralternative, the display with a quad-linear light source could have twoblocks of green light sources or emitters, such as a G1 centered at 510nm and a G2 centered at 550 nm. Alternately, the quad-linear lightsource array could be an RGB-W device, with a block of white lightsources or emitters (e.g., white light LEDs). It would then be aluminance—chrominance type display that can potentially support whiteror brighter whites than otherwise, and expand the display's dynamicrange. Having a block of white light emitters can provide greater designfreedom in the design of the color emitters, in terms of performance ormaterials selection. The linear light emitter array can be extended tohave yet further additional colors. For example, to further extend thecolor gamut, it could have six color channels, R1, R2, G1, G2, B1, andB2, with some wavelength separation between the spectral peaks orcentroids of the respective channels in a given color (e.g., R1-R2).

FIG. 28 depicts a portion of an ARHS using a design approach to reducethe size of the projection optics 2903 that are used to couple displaylight 2905 into the light guide combiner 2910. The display 2901 has amicro-lens array 2902 which has micro-lenses whose tilt is optimizedspatially over the light emitting display 2901 so as to modify theeffective light emission angle or directional tilt of the light towardsthe projection optics 2903. This result is that the overall displaylight can be better matched to the numerical aperture and acceptanceangle of the projection optics. This approach can enhance the opticalefficiency of the ARHS by removing the need for a light-absorbingaperture stop, and as help to enhance image sharpness and reduce theheadset size. In some embodiments, this micro-lens array can beconstructed as a holographic or sub-wavelength structure patterned ontop of the display.

FIG. 29 depicts another way to reduce the size of the projection optics3010 used in an ARHS. The display 3001 is combined with polarizer 3002(or, in some embodiments, the display emits polarized light directly)and the folded “pancake lens” 3003 with internal polarizing beamsplitters 3004, 3005 to reduce the length and number of discreteelements in the projection system while still maintaining high imagequality.

FIG. 30 depicts yet another way to reduce the size of the projectionoptics in an ARHS. FIG. 30 shows both a top view and a cross-sectionalview. In this example, multiple folded catadioptric lenses 3301, 3302,3303, 3304 with similar structures combine to form one discontinuousoptical element 3310 that can be used to generate a continuous imagefrom display 3305. The surfaces of these lenses can be spherical,aspheric, or free-form, and contain Fresnel, diffractive, or holographicpatterns, or have dichroic or polarization-dependent coatings. Theseelements can be used to increase the field-of-view, resolution, orsharpness of the optical system, or can be used to incorporate image,structure, or calibration sensors into the system.

FIG. 31 depicts an improved version of a ARHS of the type of FIG. 14 andFIG. 16 which uses 1D scanning and a pin-mirror light guide combiner. Inthis version, a position sensing mechanism is provided to work with themechanical scanner 3105. The light emitting display 3101 is fabricatedwith additional photodiodes 3102 in an optically inactive area. A smallpickoff mirror placed in the light path in the light guide combiner3103, after the projection optics 3104, is used to deflect a smallportion of the image light 3105 backwards through the projection optics3104, to focus onto the photodiodes. By incorporating a photodiode arraywith a diode sensor pitch smaller than the pixel pitch of the displayemitters, self-aligning position sensing with subpixel resolution can beachieved at little additional cost.

FIG. 32 depicts an improved version of a light guide combiner that canbe used in an ARHS. In this version, additional optically poweredelements are affixed to the front and back surfaces of the light guidecombiner 3203. A latching mechanism 3201 (for example, a magnet or amechanical latch) affixes one or more optical elements 3202 to thecombiner 3203. An air gap 3204 is present between the additionaloptic(s) and the combiner in order to preserve the light-guiding effectof the combiner. As the combiner alone has no optical effect on ambientrays 3205 passing through it, any effect desired from the additionaloptic is preserved. These additional optics can provide, for example,corrective or filtering functions and can be composed of geometriclenses, Fresnel lenses, diffractive optics, or electrically-controlledoptical modulators.

The present invention has been described using detailed descriptions ofembodiments thereof that are provided by way of example and are notintended to limit the scope of the invention. The described embodimentscomprise different features, not all of which are required in allembodiments of the invention. Some embodiments of the present inventionutilize only some of the features or possible combinations of thefeatures. Variations of embodiments of the present invention that aredescribed and embodiments of the present invention comprising differentcombinations of features noted in the described embodiments will occurto persons of the art.

It will be appreciated by persons skilled in the art that the presentinvention is not limited by what has been particularly shown anddescribed herein above. Rather the scope of the invention is defined bythe claims that follow.

What is claimed is:
 1. An augmented reality headset (ARHS) displayhaving a scanning projection image source comprising: a. a tri-lineardisplay with multiple rows of pixels for rendering a pixelated imagesource; b. imaging optics which provide image light to a combiner; c. ascanning system which deflects the image light along one axis; and d. acontrol system which temporally varies the image shown on each row so asto increase the perceived brightness of the image.
 2. The ARHS displayof claim 1, wherein the scanning system is a 1D scanning mirror withreciprocating motion.
 3. The ARHS display of claim 1, wherein thescanning system is a polygonal mirror rotating with uniform velocity. 4.The ARHS display of claim 1, wherein the scanning system is anelectro-optic beam deflection device.
 5. The ARHS display of claim 1,wherein the tri-linear display includes a plurality of monochromaticblocks of pixels.
 6. The ARHS display of claim 1, wherein the intensityof the pixels in each of the multiple rows of pixels in the tri-lineardisplay can be modulated and/or the number of rows of pixels in thetri-linear display which are lit can be adjusted to control a greylevel.
 7. The ARHS display of claim 1, wherein the tri-linear displayincludes one or more blocks comprising a plurality of pixels, wherein toattain a lower luminous output for a particular block the number ofpixels in a particular block are increased.
 8. The ARHS display of claim1, wherein the control system implements a pixel illumination scheme,wherein a reduction in the illumination in a column of pixels as theresult of a defective pixel in the column is compensated for byilluminating at least one additional pixel in the column.
 9. The ARHS ofclaim 1, wherein the trilinear display comprises a plurality ofmonochromatic blocks, each of the plurality of monochromatic blockscomprising a plurality of rows, and in which each of the plurality ofrows are alternately offset from adjacent rows by a fractional pixelpitch so as to enhance system resolution.
 10. The ARHS of claim 9,wherein the pixels are masked with an opaque mask so as to reduce a fillfactor.
 11. The ARHS of claim 9, further comprising a softwareanti-aliasing filter that is configured to deconvolve against a displaybox filter to create a modified anti-aliasing filter.
 12. The ARHS ofclaim 1, wherein the tri-linear display comprises more than three typesof color emitters, wherein the more than three types of color emittersexpand the color gamut of the tri-linear display.
 13. The ARHS of claim1, wherein the combiner includes an imbedded plurality of pin-mirrorsthat reflect image light towards an eye box, and wherein gaps betweenthe pin-mirrors in the plurality of pin-mirrors allow ambient light topass through the combiner to the eye box.
 14. The ARHS display of claim13, wherein a pin-mirror size, as seen by a viewer, is nominallyconstant across the combiner.
 15. The ARHS display of claim 13, whereinthe size, shape, spacing, or tilt of the pin-mirrors can be adjusted tochange the optical efficiency of the combiner directing the image lightto the eye box.
 16. The ARHS display of claim 13, wherein the size,shape, and tilt of the pin-mirrors can be adjusted to change an imageblur.
 17. The ARHS display of claim 13, wherein the size, shape,spacing, or tilt of the pin-mirrors can be adjusted to change theoptical efficiency of allowing ambient light to pass through thecombiner to the eye box.
 18. The ARHS display of claim 13, wherein acoating of the pin-mirror is configured to improve an optical efficiencyof a transiting image light or the ambient light.
 19. The ARHS displayof claim 13, wherein the plurality of pin-mirrors is arranged intomultiple sub-arrays.
 20. A method for providing an image from an ARHS,comprising; providing an ARHS with a pair of displays, each of the pairof displays comprising a pixelated color display that provides an imagelight into a corresponding projection optics; directing the image lightthrough the corresponding projection optics and edge coupling the imagelight into a lightguide combiner, wherein the lightguide combiner isconfigured to redirect transiting image light towards a correspondingeye box; wherein the light guide combiner includes an imbedded pluralityof pin-mirrors that reflect the image light towards the correspondingeye box, and wherein one or more gaps between the pin-mirrors allowambient light to pass through the combiner to the corresponding eye box;and wherein, with respect to the image light, the pin-mirrors appear toform a high-fill factor array, while simultaneously appearing as alow-fill factor array for ambient light incident to an outer sidesurface of the combiner.